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Summary 


Background 

The  inability  to  train  Chief  Engineering  Officers  (CHENGs)  to  properly  assess  the  material 
readiness  of  their  propulsion  plants  was  identified  as  a  deficiency  by  the  Chief  of  Naval 
Operations  (N869)  in  FY95.  Time,  costs,  and  material  considerations  have  limited  the  ability  of 
the  training  community  to  provide  extensive  hands-on  material  assessment  opportunities  to 
reinforce  classroom  training  conducted  in  the  Department  Head,  Prospective  Engineering  Officer 
(PEO)  course,  at  the  Surface  Warfare  Officer  School  (SWOS),  Newport,  RI.  This  deficiency  has 
been  noted  in  Operational  Propulsion  Plant  Examination  (OPPE)  results.  Department  Head 
Discharge  for  Cause  data,  and  Engineering  Training  Group  (ETG)  training  visits.  The  lack  of 
adequate  training  has  resulted  in  engineers  being  “relieved  for  cause,”  equipment  casualties,  lost 
operating  time,  and  costly  repairs.  Existing  training  capabilities  have  not  been  sufficient  to 
resolve  the  problem. 

Objective 

The  objectives  are  to:  (1)  develop  and  demonstrate  desktop  virtual  environment  (VE) 
training  for  the  prospective  engineering  officer,  (2)  select  and  demonstrate  the  instructional 
strategies  necessary  for  training  material  assessment  in  a  virtual  environment  (VE),  and  (3) 
demonstrate  the  effectiveness  of  VE  technology  in  improving  the  performance  of  Engineering 
Officers. 

Approach 

The  project  team  selected  a  case-based  approach  to  this  training  by  developing  training 
scenarios  for  use  within  a  virtual  engineering  space  that  will  provide  the  student  engineering 
officer  with  an  opportunity  to  build  on  prior  knowledge  and  experience  to  recognize  material 
discrepancies  and  take  appropriate  corrective  action. 

Project  Status 

The  team  first  obtained  and  reviewed  the  terminal  and  enabling  objectives  for  the  PEO  course 
and  for  the  “Hot  Plant”  course  at  Naval  Training  Center,  Great  Lakes,  IL.  It  was  determined  that 
material  readiness  assessment  is  not  specifically  addressed  in  the  existing  curriculum,  although 
the  tools  to  conduct  such  assessments  are  provided  in  the  curriculum.  The  team  then  reviewed 
recent  Propulsion  Examination  Board  (PEB)  reports  with  the  assistance  of  personnel  from 
Instructional  Science  and  Development,  Inc.  A  thorough  review  of  the  capabilities  and 
limitations  of  available  virtual  environment  technologies  was  then  conducted.  Immersive 
environments  were  rejected  in  favor  of  the  non-immersive  desktop  personal  computer  (PC) 
environment  due  to  insufficient  fidelity  and  the  potential  adverse  effects  of  motion  sickness  in 
the  immersive  environment.  Navigation  through  a  three-dimensional,  modeled  engineering 
space  on  the  PC  was  adequate  in  light  of  the  requirement  for  the  engineering  officer  to  navigate 
and  remain  oriented  in  the  engineering  spaces.  Although  the  fidelity  was  still  less  than  optimal 
to  meet  a  requirement  to  read  dials,  gauges,  tags,  stencils,  etc.,  it  was  still  better  than  that 
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available  in  head-mounted  displays  (HMDs).  In  addition,  the  PC  had  greater  ease  of  use  and 
lower  cost. 

Research  Triangle  Institute  (RTI)  was  contracted  to  develop  a  three-dimensional  modeled 
environment  based  on  the  3D  computer-assisted  design  (CAD)  drawings  of  the  Arleigh  Burke 
class  destroyer  (DDG51)  obtained  from  a  third  party  contractor.  The  technology  to  do  this  did 
not  exist  at  the  outset  of  this  project.  As  the  state  of  technology  improved  over  the  life  of  the 
program,  RTI  was  able  to  create  a  second  rendering  of  the  environment  using  2D  digital  images 
taken  from  a  physical  DDG51  propulsion  plant.  Discrepancies  could  be  “painted”  into  the  digital 
environment.  It  also  became  possible  to  incorporate  the  content  of  some  2D  drawings  into  the 
original  modeled  environment.  Measures  of  performance  (MOP)  and  measures  of  effectiveness 
(MOE)  for  material  assessment  training  were  identified,  and  two  initial  training  scenarios  were 
created.  These  incorporated  the  common  discrepancies  found  in  the  PEB  review.  With 
instructor  modeling  and  guidance,  the  student  will  identify  and  prioritize  these  discrepancies  and 
propose  a  plan  of  action  to  resolve  each  one.  It  is  hypothesized  that  students  using  the  desktop 
VE  to  perform  material  assessment  in  a  case-based  setting  will  perform  better  than  students  using 
either  conventional,  paper-based  visuals  alone  or  conventional,  paper-based  visuals  with  a  case- 
based  approach  to  material  assessment. 

An  evaluation  plan  was  developed,  and  the  results  will  be  analyzed  both  quantitatively  and 
qualitatively.  A  draft  life  cycle  support  plan  was  also  produced.  The  evaluation  plan  is  ready  to 
be  executed.  It  is  anticipated  the  evaluation  can  be  completed  within  FY98. 

Recommendations 

This  product  should  be  evaluated  and  recommendations  provided  to  SWOS,  N869,  and 
PERS-OOH.  The  digital  imagery  technology  shows  promise  for  the  development  of  inexpensive 
training  for  environments  where  complex  equipment  with  significant  safety  and  operational 
requirements  precludes  use  of  actual  systems.  The  modeled  technology  should  be  further 
developed  as  improved  visual  resolution  becomes  available  for  low-cost  training  delivery 
systems. 

Status  to  Date 

The  VET-E  project  was  transferred  in  its  final  year  to  NAWCTSD,  Orlando,  FL.  The 
transfer  is  the  result  of  the  NPRDC  Training  Department  function  being  transferred  to 
NAWCTSD  as  a  consequence  of  BRAC  95.  The  foregoing  materials  describe  the  background 
and  status  of  the  project  as  of  the  5th  of  December,  1997.  The  future  direction  and  development 
of  the  project  will  be  determined  by  NAWCTSD,  in  concert  with  the  membership  of  the  IPG. 
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Introduction 


Problem 

The  inability  to  train  Chief  Engineering  Officers  (CHENGs)  to  properly  assess  the  material 
readiness  of  their  propulsion  plants  was  identified  as  a  deficiency  by  the  Chief  of  Naval 
Operations  (N869)  in  FY95.  Time,  costs,  and  material  considerations  have  limited  the  ability  of 
the  training  community  to  provide  extensive  hands-on  material  assessment  opportunities  to 
reinforce  classroom  training  at  the  Surface  Warfare  Officer  School  (SWOS),  Newport,  RI.  This 
deficiency  has  been  noted  in  Operational  Propulsion  Plant  Examination  (OPPE)  results. 
Department  Head  Discharge  for  Cause  data,  and  Engineering  Training  Group  (ETG)  training 
visits.  The  lack  of  adequate  training  has  resulted  in  engineers  being  “relieved  for  cause.” 
equipment  casualties,  lost  operating  time,  and  costly  repairs.  Existing  training  capabilities  have 
not  been  sufficient  to  resolve  the  problem. 

Objective 

The  objective  was  to  experimentally  evaluate  the  feasibility  of  using  case-based  training 
methods  with  a  “desktop”  virtual  environment  to  conduct  material  readiness  assessment  training 
for  Prospective  Engineering  Officers  (PEOs)  in  the  Department  Head  course  at  SWOS,  Newport, 
RI. 

Background 

An  N969  requirement,  in  FY95,  directed  the  development  of  a  training  program  that  provides 
(1)  a  simulation  of  the  propulsion  plant  environment  to  provide  hands-on  material  assessment 
capability  in  the  schoolhouse  and  (2)  a  training  methodology  that  uses  engineering  casualty 
experiences  that  identify  the  material  readiness  condition  of  the  propulsion  plant. 

Implementation  Planning  Group 

An  Implementation  Planning  Group  (IPG)  was  established  by  the  N869  Project  Officer  in 
FY95.  The  IPG  ensures  that  the  research  effort  remains  responsive  to  a  valid  requirement  and 
serves  as  the  planning  vehicle  by  which  the  transition  of  products  developed  through  the 
advanced  development  process  reach  operational  use.  IPG  responsibilities  and  membership  are 
outlined  by  the  OP-01  MPT  Research  and  Development  Project  Officer’s  Guidebook.  The  IPG, 
chaired  by  N869T4,  included  Navy  Personnel  Research  and  Development  Center 
(NAVPERSRANDCEN  Code  131);  Bureau  of  Personnel  (PERS-OOH);  Chief  of  Naval  Education 
and  Training  (CNET  T23);  SWOS,  Code  61;  Naval  Sea  Systems  Command 
(NAVSEASYSCOM  042);  NAVSEASYSCOM  (PMS  400  F32);  Commander  Naval  Surface 
Force  Pacific  (COMNAVSURFPAC  N813);  and  Office  of  Naval  Research  (ONR  342B). 

Material  Assessment 

To  ensure  full  compliance  with  their  duties,  as  specified  in  the  Standard  Organization  and 
Regulations  of  the  U.S.  Navy,  and  elsewhere,  Engineering  Department  personnel  are  responsible 
for  identifying  and  correcting  equipment  deficiencies  in  their  spaces.  Frequently  this  takes  the 
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form  of  a  “walk-through”  of  the  engineering  spaces  in  response  to  specific  problems  and  to 
determine  the  status  of  work  in  progress.  Examples  of  material  readiness  deficiencies  include, 
but  are  not  limited  to,  gauges  reflecting  out-of-parameter  conditions,  excessive  pump  leak  off, 
blown  gaskets,  and  safety  hazards. 

Technologies 

A  case-based  instructional  strategy  combined  with  a  “desktop”  virtual  environment  was 
selected  to  deliver  material  assessment  training.  The  training  problem  is  not  limited  to  the 
Prospective  Engineering  Officer’s  timely  assessment  of  the  material  condition  of  the  propulsion 
plant.  Retention  of  what  has  been  learned  is  also  an  issue  given  the  limited  opportunity  to 
practice  under  realistically  simulated  conditions.  In  work  reported  by  Semb  and  Ellis  (1994),  it 
was  found  that  instructional  strategies  that  actively  engage  students  result  in  better  retention. 
Case-based  learning  (CBL)  instructional  approach,  properly  designed  and  delivered  in  a  virtual 
environment,  should  actively  engage  the  students  and  result  in  the  needed  retention. 

Virtual  Environments 

Time,  cost,  and  material  considerations  limit  the  ability  of  the  training  community  to  provide 
sufficient  “hands  on”  experience  with  simulated  material  assessment  opportunities  to  reinforce 
classroom  training.  Training  delivered  in  a  virtual  environment  (VE)  should  provide  a  solution 
to  the  problem  of  limited  availability  of  training.  Also,  current  training  limits  the  scope  of 
material  discrepancies  that  CHENGs  can  encounter  due  to  safety  concerns.  Introducing  a  fault 
into  an  operating  turbine  engine  that  could  result  in  catastrophic  failure  is  not  a  viable  training 
option.  Given  projections  for  desktop  computer  power  and  display  systems  to  increase  in 
performance  while  simultaneously  decreasing  in  cost,  PC -based  “desktop”  virtual  environment 
training  delivery  systems  will  be  efficient  solutions  in  the  near  to  mid  term.  There  are  also 
opportunities  for  gains  in  training  availability.  New  laptop  PCs  are  increasing  in  capability  and 
decreasing  in  cost,  providing  an  opportunity  to  make  the  “desktop”  VE  training  available  not 
only  in  the  schoolhouse,  but  also  while  deployed,  thereby  overcoming  a  major  limitation  of 
current  training  technologies. 

Case-Based  Learning 

CBL,  also  known  as  scenario-based,  uses  accumulated  prior  experience  to  prepare  personnel 
to  recognize,  understand,  and  solve  new  problems.  This  model  proposes  that  previous  situations 
relevant  to  the  current  engineering  problem  are  recalled,  compared  with  the  current  propulsion 
plant  situation,  and  through  a  process  of  interpretation  and  generalization  lead  to  a  solution  for 
the  current  problem.  The  use  of  a  CBL  environment  would  allow  PEOs  to  experience  numerous 
scenarios  containing  discrepant  equipment,  thereby  building  up  an  extensive  set  of  previous 
situations  for  recall.  As  a  training  methodology,  it  provides  modeling  of  the  desired  behaviors 
and  approaches  to  learning  and  problem  solving,  feedback  on  performance,  and  an  orientation  for 
decision-making.  It  was  selected  for  use  in  this  effort  since  it  closely  approximates  the  working 
(performance)  environment  of  the  CHENG  aboard  ship. 
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Approach 


The  approach  used  in  the  VET-E  project  consisted  of  several  phases:  The  definition  of  the 
tasks  the  CHENG  performs;  the  identification  and  conversion  of  an  appropriate  shipboard 
propulsion  plant  into  a  virtual  environment;  the  review  of  the  existing  Prospective  Engineering 
Officer  Course  and  Curriculum;  identification  of  specific  areas  of  performance  deficiency; 
assessment  of  available  VE  technologies  and  applications;  identification  and  assessment  of 
appropriate  instructional  strategies;  integration  of  the  VE  technologies  with  the  most  appropriate 
instructional  strategy;  and  design  and  development  of  an  evaluation  plan  for  assessing  the 
effectiveness  of  the  resultant  training  program  and  devices. 

Definition  of  CHENG  Tasks 

Cognitive  task  interviews  were  conducted  to  define  the  engineering  tasks  performed  by  the 
CHENG.  The  interviewees  consisted  of  five  personnel  from  the  Propulsion  Engineering  Board 
(PEB)  who  had  served  an  average  of  17  months  as  Chief  Engineering  Officer  (CHENG)  and  19.5 
months  as  a  Main  Propulsion  Assistant  (MPA);  five  instmctors  from  the  Department  Head 
Prospective  Engineering  Officer  (PEO)  course  at  Naval  Training  Center  (NTC),  at  Naval  Station 
(NS)  Great  Lakes,  four  of  whom  were  Engineering  Officer  of  the  Watch  (EOOW)  qualified  and 
had  served  an  average  of  23  months  in  an  Engineering  Department  billet;  three  instructors  from 
the  Marine  Propulsion  course  at  SWOS;  two  personnel  from  the  Afloat  Training  Group  (ATG) 
San  Diego,  one  CHENG  and  one  MPA. 

The  interviewees  described  the  CHENG  position  as  one  of  high  stress  and  very  high 
workload  within  a  complex  work  environment  requiring  good  management  skills  as  well  as 
attention  to  detail.  A  CHENG  must  have  a  well  qualified  team  and  must  understand  the 
operation  of  the  engineering  systems.  Working  with  limited  resources,  a  CHENG  must  be  able 
to  prioritize  which  deficiencies  will  receive  the  Engineering  Department’s  primary  attention. 
CHENGs  also  have  responsibilities  not  directly  related  to  the  Engineering  department  which 
require  their  attention,  thereby  reducing  the  amount  of  time  they  can  spend  working  on  purely 
engineering  duties.  It  is  in  this  work  environment  that  the  task  of  material  assessment  must  be 
performed.  The  preparation,  prior  experience,  and  training  of  the  PEO  is  shown  in  Figures  1  and 
2. 


At  its  most  basic  level,  material  assessment  is  the  CHENG’s  perception  of  the  variables  in 
the  engineering  environment.  These  variables  may  be  the  operating  parameters  of  a  system,  the 
“proper  feel”  of  lube  oil,  or  the  “squared  away”  look  of  a  well  maintained  engineering  space. 
This  recognition  process  is  akin  to  the  first  step  in  what  Klein  (1989)  has  called  Recognition 
Primed  Decision  (RPD)  making  as  well  as  what  Endsley  (1995a,  1995b)  calls  Situation 
Awareness  (S  A).  However,  both  RPD  and  S  A  go  on  to  include  behavior  after  the  recognition  (or 
assessment)  of  the  environment  such  as  what  actions  to  take  to  reach  certain  goals. 
Differentially,  Klein  generally  is  describing  a  process  whereas  Endsley  generally  is  describing  a 
state. 
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Figure  1.  Training  and  experience  for  prospective  engineering  officers. 


Figure  2.  Surface  Warfare  Officers  School  Department  Head  Training 


Generally,  interviewees  described  three  primary  means  of  assessing  whether  or  not  the 
various  engineering  systems  are  operating  within  acceptable  limits:  (1)  Visual  inspection, 
augmented  by  sound  and  touch.  A  majority  of  respondents  when  asked,  “What  is  material 
assessment?”  responded  by  describing  some  form  of  visual  inspection,  calling  it  variously 
“housekeeping”  or  space  cleanliness,  or  “Just  the  appearance  of  the  spaces.”  (2)  Equipment  logs, 
which  allow  trends  to  be  developed  over  time  for  particular  systems  or  subsystems.  These  logs 
provide  both  a  source  of  data  and  a  structure  to  organize  that  data.  Engineering  Operating 
Sequencing  System  (EOSS)  and  Engineering  Operating  Procedures  (EOP)  are  additional  sources 
of  “structure”  that  are  used  by  the  CHENG  to  organize  their  operational  environment  and  ensure 
that  the  engineering  plant  is  functioning  properly.  (3)  Discrepancies  discovered  through  routine 
maintenance  checks  such  as  those  performed  according  to  the  Planned  Maintenance  System 
(PMS).  Many  of  the  former  CHENGs  related  having  to  rely  more  on  the  logs  and  PMS  checks 
because  they  found  it  difficult  to  find  the  time  to  conduct  frequent  walkthroughs  of  the 
engineering  spaces.  They  relied  on  the  other  members  of  the  engineering  team  to  perform 
frequent  visual  inspections.  This  reliance  on  other  team  members  underscores  the  importance  of 
having  a  well-trained  and  experienced  Engineering  Department  which  is  not  always  the  case. 
Training  and  experience  play  a  large  part  in  properly  performing  the  maintenance  checks. 
CHENGs  reported  the  logs  presented  some  problems  simply  due  to  the  sheer  number  of  logs  and 
the  amount  of  data  in  each  log  that  had  to  be  reviewed. 

The  two  most  common  responses  as  to  why  some  CHENGS  had  difficulty  performing 
material  assessment  were:  (1)  Lack  of  system  knowledge,  resulting  in  ignoring  perceived  minor 
discrepancies  that  were  in  fact  symptomatic  of  major  problems  and  (2)  inexperience  in  managing 
the  various  aspects  of  the  PMS  so  that  maintenance  was  either  not  performed  or  performed 
incorrectly.  Several  respondents  also  noted  that  often  a  perceived  minor  discrepancy  does  not  get 
corrected  immediately.  The  more  time  passes,  the  more  the  discrepancy  begins  to  look  like  part 
of  the  “normal  environment”  so  that  after  awhile  people  forget  that  it  actually  is  a  discrepancy. 
Other  factors  were  also  reported  to  cause  equipment  to  be  left  in  a  discrepant  state.  Only  when 
an  inspection  team  points  it  out  or  a  significant  malfunction  occurs  because  of  the  existing 
discrepancy  does  it  become  “visible”  again.  Unwillingness  to  report  needed  equipment  repairs 
because  the  ship  may  not  be  able  to  perform  its  mission,  inadequate  funding  to  effect  needed 
repairs,  and  an  inability  to  manage  people  effectively  also  were  reported  by  one  or  more  of  the 
respondents. 

Identification  and  Conversion  of  Plant  Layout  into  Virtual  Environment 

Computer-assisted  design  (CAD)  drawings  for  the  DDG51,  Arleigh-Burke  class,  were 
obtained.  These  CAD  blueprints  were  converted  into  computer  graphics  files,  which  were  then 
converted  into  basic  two-dimensional  and  three-dimensional  computer  files.  The  conversion  of 
these  basic  drawings  served  as  the  foundation  for  “building”  or  rendering  an  environment  that 
modeled  the  DDG51  propulsion  plant. 

Additionally,  the  actual  DDG51  propulsion  plant  environment  was  photographed  from 
multiple  locations  on  the  three  levels  in  the  ship.  At  each  location,  photos  were  taken  around  a 
center  point  to  provide  a  360  degree  view.  These  digital  photo  images  were  combined  or 
“stitched”  together  to  present  a  panorama  of  the  locations  photographed.  Subsequently,  it  was 
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discovered  that  material  readiness  “discrepancies”  could  be  graphically  “painted”  into  these 
images  to  create  the  appearance  of  a  problem  in  the  environment. 

Review  of  Prospective  Engineering  Officer  Course 

PEO  Course  Review 

A  review  of  the  PEO  course  was  contracted.  The  training  objectives,  both  terminal  and 
enabling,  were  identified  and  reviewed.  These  objectives  focused  on  locating  and  identifying 
components  in  the  propulsion  and  auxiliary  systems  as  well  as  describing  the  interrelationships 
between  various  propulsion  systems  and  the  engineering  plant  as  a  whole.  The  results  of  the  PEO 
course  review,  interviews  with  the  Propulsion  Examining  Board,  and  the  initial  training  scenarios 
are  contained  in  the  report  prepared  by  Instructional  Science  and  Development,  Inc.  (ISD) 
(March,  1996),  provided  in  Appendix  A.  The  training  scenarios  subsequently  were  revised  and 
lengthened.  The  revised  scenarios  were  reported  by  ISD  in  December,  1996,  and  are  provided  in 
Appendix  B.  The  final  scenarios,  drawn  from  the  earlier  efforts  with  a  script  incorporating  a 
case-based  approach  to  instruction,  are  found  in  Appendix  C. 

At  SWOS,  students  are  provided  classroom  instmction  on  what  to  look  for  in  material 
readiness  assessment.  After  completing  their  training,  optionally,  some  students  proceed  to  NTC 
Great  Lakes  enroute  to  their  shipboard  assignment.  At  NTC  Great  Lakes,  they  undergo  one  to 
two  weeks  of  training  on  the  “hot  plant,”  an  operating  propulsion  plant  used  for  familiarization 
and  demonstration  purposes.  There  they  are  encouraged  to  identify  material  deficiencies. 
Instructors  at  the  hot  plant,  however,  are  unable  to  introduce  realistic  fault  conditions  into  the 
equipment  due  to  safety  and  cost  considerations.  Thus,  it  is  difficult  to  realistically  simulate 
material  deficiencies  to  reinforce  their  earlier  classroom  lectures.  In  general,  the  two  week 
course  was  found  to  be  informal;  a  “gentleman’s  course”  was  one  instructor’s  comment.  When 
instructors  were  asked  how  they  knew  who  was  performing  adequately  and  who  needed  help 
they  gave  anecdotal  evidence:  “You  can  tell  who  needs  help  and  who  doesn’t  by  the  questions 
they  ask,”  one  instructor  noted  when  asked  about  student  performance.  These  findings  suggest 
the  need  to  incorporate  specific  training  objectives  and  instruction  into  the  course  work  presented 
at  SWOS  and  at  the  hot  plant. 

Interviews  with  Propulsion  Examination  Board  (PEB)  Members 

In  addition  to  the  curriculum  objectives  required  for  training,  interviews  were  conducted  with 
subject  matter  exerts  to  identify  the  skills  and  knowledge  needed  for  Chief  Engineering  Officers 
to  be  successful  in  assessing  the  material  condition  of  their  propulsion  plants.  The  results  of 
these  interviews  identified  three  primary  areas  of  expertise.  The  first  and  most  basic  is  (1) 
pattern  recognition.  Pattern  recognition  is  the  ability  to  identify  an  object  as  a  member  of  a  class 
(Watanabe,  1985).  For  example,  an  experienced  CHENG  can  identify  whether  or  not  a  flange 
shield  is  in  the  class  of  “discrepant  flange  shields.”  This  skill  is  built  up  over  time  through 
experience  with  similar  equipment  and  problems  in  the  engineering  spaces.  (2)  System 
knowledge  is  the  second  area  of  importance  identified.  A  CHENG  may  not  be  able  to  tell  if 
something  is  discrepant  unless  he  understands  what  role  that  object  plays  in  the  larger 
engineering  system.  A  lack  of  understanding  of  the  interrelationships  that  exist  in  the  complex 
engineering  systems  was  seen  as  a  common  problem.  This  knowledge  base  is  built  up  through 
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formal  schooling  and  experience.  (3)  Management  skill  is  the  third  area  of  necessary  expertise 
for  a  CHENG.  A  CHENG  must  be  able  to  organize  the  ship’s  engineering  resources  to  properly 
monitor  and  maintain  the  engineering  equipment.  This  includes  using  his  time  efficiently  since  a 
CHENG  has  many  duties  other  than  engineering  responsibilities.  This  expertise  is  also  built  up 
through  formal  schooling  and  job  experience. 

Identify  Specific  Areas  of  Deficiency 

To  determine  the  most  common  readiness  assessment  problems  that  occurred  for  CHENGs, 
an  assessment  was  made  of  the  Operational  Propulsion  Plant  Examination  (OPPE)  reports. 
These  reports  are  prepared  by  inspectors  whose  jobs  it  is  to  identify  problem  areas  aboard  ship, 
or  by  Subject  Matter  Experts  (SMEs).  The  results  of  this  analysis  are  contained  in  the  report 
prepared  by  Instructional  Science  and  Development,  Inc.  (ISD)  (March,  1996),  found  in 
Appendix  A. 

To  construct  cases  or  scenarios  that  would  be  relevant  to  Fleet  engineering  officers  the 
material  readiness  sections  of  OPPE  reports  were  used  to  determine  what  equipment  was  most 
problematic  aboard  ship.  The  areas  of  interest  were  Main  Engine  Room  #1  and  Main  Engine 
Room  #2.  Some  61  OPPE  reports  from  both  Commander  Naval  Surface  Force  Pacific 
(COMNAVSURFPAC)  and  Commander  Naval  Surface  Force  Atlantic  (COMNAVSURFLANT) 
were  reviewed.  An  example  of  a  list  of  equipment  from  a  frigate  (FFG)  and  a  cruiser  (CG)  noted 
to  have  material  deficiencies  is  shown  in  Figure  3.  An  “X”  under  the  ship  class  indicates  an 
occurrence  of  the  equipment  as  deficient  in  an  OPPE  report. 


Main  Engine  Room 

FFG 

CG 

HP  Air  Reducing  Valve 

X 

X 

Lube  Oil  Unloader  Pressure 

X 

Start  Air  Reducing  Valve  Leak 

X 

X 

Figure  3.  Examples  of  equipment  deficiencies. 

The  list  of  equipment  developed  from  the  OPPE  reports  was  given  to  a  SME  to  construct  a 
“walkthrough”  scenario  that  would  focus  on  the  equipment  that  was  noted  frequently  in  the 
OPPE  reports.  Initially,  two  scenarios  were  developed  and  given  to  PEB  members  and  to  NTC 
Great  Lakes  instructors  for  feedback  and  revisions.  Subsequently,  it  was  decided  that  the  project 
testbed  would  be  at  SWOS  in  the  PEO  course.  As  a  result,  these  scenarios  were  also  reviewed 
and  revised  by  SWOS  participants.  The  revised  scenarios,  found  in  Appendix  C,  will  be  used  as 
the  basis  for  the  case-based  instructional  scripts  and  the  evaluation  of  the  present  project. 
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Assess  Existing  VE  Technologies  and  Applications 


The  NPRDC  research  team  visited  a  number  of  sites,  both  commercial  and  academic,  to  gain 
a  better  understanding  of  the  current  state  of  VE  development.  This  was  done  prior  to  awarding 
a  contract  for  development  of  the  software  environment.  Sites  visited  included: 


University  of  New  Mexico 
Sandia  National  Laboratories 
Lockheed  Martin 
Southwest  Research  Institute 
Research  Triangle  Institute 


Armstrong  Lab,  Brooks  AFB 
General  Dynamics,  Electric  Boat  Division 
Naval  Research  Lab  (NRL) 

Naval  Surface  Warfare  Center,  White  Oak 
Kaiser  Electro-Optics,  Inc. 


The  developmental  options  included  whether  to  use  a  head  mounted  display  (HMD)  and  an 
audio  format  to  localize  engine  room  sounds.  An  HMD  was  not  used  because  of  insufficient 
resolution  for  displaying  the  material  readiness  deficiencies  and,  secondarily,  potential  simulator 
sickness.  As  HMD  resolution  improves,  this  decision  should  be  revisited.  Audio  effects  were 
obtained  for  64  different  engine  room  sounds.  But,  time  and  cost  considerations  did  not  allow 
incorporating  them  into  the  engine  room  environments  for  the  feasibility  demonstration. 

Two  versions  of  the  desktop  VE  were  developed:  A  3-D  modeled  environment  and  a  digital 
imagery  environment.  The  modeled  environment  was  constructed  using  the  original  DDG51 
CAD  files  which  provided  the  structural  aspects  of  the  ship.  These  were  translated  into  a  GAP 
graphics  format  and  then  to  a  DXF  format,  which  was  then  imported  into  3D  Studio.  The 
structural  or  wire  frame”  view  is  shown  in  Figure  4.  The  equipment  in  the  modeled  engine 
room  environment  was  built  using  3D  Studio,  Release  4.  A  Microsoft  Visual  C-h-  application 
provided  the  navigational  capability  within  the  environment  and  SenseS’s  WorldToolKit  3-D 
provides  the  rendering  API.  WorldToolKit  is  "the  software  of  choice  for  PC-based  VR 
applications  because  of  its  support  for  nearly  cill  the  VR  peripherals  on  the  market  today" 
(Phillips,  1995,  p.  3).  The  PC-based  pentium  equipment  required  to  support  the  two 
environments  are  found  in  Appendix  D. 


Main  Engine  Room  #2,  Platforms  1,  2,  and  the  Hold,  were  constructed  using  3-D  models. 
Originally,  both  Main  Engine  Rooms  #1  and  #2  were  to  be  fully  modeled  in  3-D  for  the  desktop 
VE,  but  resource  constraints  did  not  allow  it.  Figure  5  shows  a  view  from  the  modeled 
environment  which  was  built  up  from  the  wire  frame  model. 
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Figure  4.  "Wire  Frame”  view  from  the  DDG51  modeled  environment 


Figure  5.  View  of  the  modeled  DDG51  engine  room. 
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The  digital  image  environment  was  constructed  from  a  series  of  individual  digital 
photographs  of  the  engineering  spaces  aboard  a  DDG51  Class  ship.  These  individual  digital 
photographs  were  “stitched”  together  using  Apple’s  QuickTime  VR  stitcher  to  provide  360° 
panoramas.  Each  panorama  is  referred  to  as  a  node.  Both  engine  rooms  exist  in  the  digital 
image  environment  as  described  above.  An  example  of  the  view  from  a  node  is  shown  in  Figure 


Figure  6.  Digital  image  environment  view  of  DDG51  engine  room  with  schematic. 

The  panoramas  from  each  node  are  embedded  into  a  MacroMedia  Director  application.  The 
floor  plan  to  the  right  of  the  scene  allows  the  user  to  navigate  in  the  digital  engineering  space. 
Each  large  black  dot  represents  a  node.  In  the  actual  system  the  unoccupied  node  dots  appear 
blue  while  the  occupied  or  viewpoint  node  appears  red.  Midway  through  the  effort  it  was 
decided  to  use  the  digital  VE  version  as  opposed  to  the  modeled  version.  It  had  become  clear 
that  the  resolution  necessary  to  display  the  more  subtle  engineering  discrepancies  could  not  be 
achieved  with  the  technology  available.  The  necessary  resolution  should  be  available  in  the 
future,  however,  and  the  next  generation  system  should  revisit  this  decision. 

Identify  and  Assess  Appropriate  Instructional  Strategies 

The  three  areas  of  expertise  necessary  to  perform  material  readiness  assessment  (pattern 
recognition,  system  knowledge,  and  management  skills)  are  enhanced  through  practice  involving 
problem  solving,  critical  thinking,  and  reasoned  judgment.  These  skills  can  be  taught  or 
reinforced  through  the  case-based  learning  approach.  Learning  results  from  the  process  of 
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resolving  a  problem  embedded  within  a  case.  The  cases  or  scenarios  in  this  project  use  the 
instructor  as  a  model  for  the  assessment  process.  The  cases  have  specific  objectives  and  a 
restricted  field  of  student  options  for  exploration  within  the  problem  environment. 

The  presentations  for  this  project  identify  material  conditions  in  Main  Engine  Room  2  of  an 
Arleigh  Burke  class  ship.  The  specific  discrepancies,  however,  are  generic  discrepancies  based 
on  the  results  of  the  OPPE  reports  and  interviews  with  SMEs  (see  Appendices  A  and  B).  The 
scenarios  replicate  performance  of  a  routine  task  in  a  dangerous,  complex  environment. 

Integrate  the  VE  Technologies  With  the  Most  Appropriate  Instructional  Strategy 

The  intent  of  the  VET-E  project  is  to  produce  a  high  fidelity  desktop  virtual  environment  in 
which  a  case-based  learning  opportunity  is  present.  A  case-based  instructional  strategy  was 
selected  as  it  most  closely  replicated  the  working  or  performance  environment  for  the  CHENG. 
The  cases,  as  used  here,  are  notional  or  composite  cases  built  from  discrepancies  identified  by  the 
PEB,  SWOS  instructors,  and  casualty  reports.  They  are  not  historical  cases,  although  the 
addition  of  selected  historical  cases  would  be  appropriate  following  the  feasibility  demonstration 
and  as  the  students  gain  experience. 

Design  and  Develop  an  Evaluation  Plan  for  VET-E 

The  objective  of  the  evaluation  is  to  demonstrate  the  effectiveness  of  the  desktop  virtual 
environment  (VE)  training  program  to  train  material  readiness  assessment.  The  training  program 
is  presented  in  the  form  of  a  scenario  made  up  of  a  number  of  discrepancies  in  the  engineering 
spaces  aboard  a  DDG51  class  ship. 

Evaluation  Plan  for  Virtual  Environment  for  Engineering 

Objective 

The  objective  of  the  evaluation  is  to  demonstrate  the  effectiveness  of  the  virtual  environment 
training  program  to  train  material  readiness  assessment. 

Training  Program 

The  VE  training  program  is  presented  as  a  scenario  made  up  of  a  number  of  discrepancies  in 
the  engineering  spaces  aboard  ship.  The  training  contains  two  elements:  (1)  an  image-based 
interactive  computer  graphics  program  portraying  the  engineering  spaces  and  equipment  aboard 
ship;  and  (2)  a  case-based  approach  (Williams,  1992)  to  teaching  material  assessment  which 
includes  instructor  modeling  of  the  material  assessment  process,  instructor  coaching  of  the 
students,  student  practice  with  the  computer  program,  and  feedback. 

Elements  (1)  and  (2)  together  are  an  appropriate  integrated  approach  to  the  subject  and 
constitute  the  experimental  approach.  Those  receiving  the  training  described  above  are  the 
experimental  group.  Group  C. 
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To  test  the  value  of  the  experimental  approach,  two  control  groups  will  be  set  up  for 
comparison  with  the  experimental  group.  The  two  control  groups  are: 

Group  A.  Photographs  of  the  engineering  spaces  and  equipment  will  be  provided  with  a 
paper  scenario.  The  engineering  spaces  are  divided  into  nodes  or  locations  where  photos  were 
taken.  They  are  based  on  the  same  digital  images  used  in  the  computer  graphic  process.  Eight 
contiguous  photos  are  presented  per  node.  A  total  of  34  nodes  will  be  used,  half  for  each 
scenario.  The  subjects  will  be  told  to  find  the  discrepancies  in  the  scenario  using  the 
photographs. 

Group  B.  This  group  will  be  given  the  same  materials  as  Group  A,  but  will  also  be  trained 
using  the  same  case-based  approach  as  the  experimental  group.  Group  C. 

The  case-based  approach  includes: 

1.  Mini-cases.  A  mini-case  is  a  segment  drawn  from  a  larger  case  (Spiro  &  Jehng,  1990). 
Each  of  the  discrepancies  is  an  instance  of  a  problem  that  could  occur  aboard  ship.'  It  is  a 
concrete  incident  of  what  could  be  found  while  doing  an  inspection  of  the  engineering  spaces.  A 
scenario  consists  of  a  series  of  mini-cases  or  discrepancies* 

2.  Instructor  modeling  of  the  material  assessment  process.  Modeling  consists  of  a 
demonstration  by  the  instructor  of  how  he  performs  material  assessment.  While  the  instructor 
models  the  process,  s/he  explains  her/his  reasoning  to  the  students.  Included  in  the  discussion  are 
the  identity  of  the  discrepancy,  the  corrective  action  to  be  taken,  the  urgency  of  the  problem,  and 
an  assessment  of  the  situation  as  affected  by  this  particular  event,  including  system  interactions. 

3.  Instructor  coaching  of  the  students.  After  the  modeling  process  is  complete,  coaching 
takes  place.  Coaching  involves  the  instructor  observing  students  as  they  try  to  complete  tasks  and 
providing  hints  and  helps  when  needed. 

4.  Student  practice  with  the  computer  application.  Following  a  period  of  coaching,  the 
student  looks  for  discrepancies  without  the  aid  of  the  instructor. 

5.  Feedback.  When  the  student  completes  his/her  assessment  s/he  returns  to  the  instructor  to 
get  feedback  on  the  results. 

The  first  two  or  three  steps  of  the  case-based  instruction  could  be  completed  with  a  group  of 
two  to  three  students  with  an  instructor.  The  instruction  is  delivered  from  a  script  to  ensure 
uniformity  of  the  instruction  across  groups.  (See  Appendix  C.) 


For  each  scenario,  testing  scenario  and  evaluation  scenario,  the  discrepancies  were  selected 
from  a  pool  of  discrepancies  so  that  the  difficulty  of  each  scenario  would  be  about  equal.  The 
discrepancies  were  rated  on  a  scale  of  1  to  3  for  criticality.  The  following  criteria  were  used  to 
select  the  discrepancies  for  each  of  the  scenarios: 

1.  One  discrepancy  of  high  criticality  (1),  two  discrepancies  of  intermediate  criticality  (2), 
and  two  discrepancies  of  lowest  criticality  (3)  were  used  for  each  scenario. 
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2.  Discrepancies  were  evenly  distributed  across  the  three  platforms  of  the  ship  (Platform  1, 
Platform  2,  Hold). 

3.  More,  than  one  discrepancy  was  present  at  some  nodes  (areas)  so  that  the  student  would 
practice  a  complete  assessment  of  the  spaces. 

4.  Some  nodes  have  no  discrepancies. 

5.  Each  scenario  includes  documented  problems  that  have  occurred  on  ships. 

6.  Discrepancies  include  (1)  system  knowledge,  (2)  management  issues,  (3)  and  visual 
recognition  components. 

7.  A  time  limit  of  45  minutes  was  initially  set  for  each  scenario.  However,  after  pilot 
testing,  the  time  was  increased  to  one  hour. 

Subjects 

Prospective  Engineering  Officers  (PEOs)  attending  the  Department  Head  course  at  Surface 
Warfare  Officer’s  School  (SWOS)  will  be  used  for  the  experimental  training  and  evaluation. 
Courses  convene  six  times  a  year,  and  each  class  contains  approximately  20  PEOs,  10-15  of 
whom  are  gas  turbine  specialists.  At  least  10  subjects  will  be  mn  in  each  of  the  three  groups, 
more  if  available.  Subjects  will  be  obtained  approximately  one  week  before  the  conclusion  of 
each  class,  starting  with  the  February  1998  class,  and  continuing  with  the  May  and  July  classes. 

Instructors 

For  the  two  group  that  are  receiving  the  case-based  instruction  (Groups  B  and  C),  two  or 
three  instructors  will  be  used.  The  same  instructors  should  be  used  for  both  groups  to  ensure  the 
equality  of  the  instmction.  It  is  recommended  that  the  instructors  be  from  SWOS  to  ensure 
familiarity  with  the  subject  matter.  The  instructors  will  practice  the  script  with  other  test  subjects 
to  ensure  they  are  equally  proficient  with  both  groups  receiving  the  instruction. 

Design 

The  testing  phase  will  be  carried  out  using  a  scenario  different  from  that  used  during  the 
training.  All  groups  will  have  completed  the  Prospective  Engineering  Officer  course  and  will 
have  had  the  standard  SWOS  training  in  the  subject  matter.  All  groups  will  spend  approximately 
the  same  time  with  the  training  scenario.  Group  A  will  be  given  no  additional  instruction  beyond 
that  received  in  the  classroom.  Group  B  will  be  trained  using  the  case-based  approach.  Group  C 
will  be  trained  in  the  case-based  approach  with  the  benefit  of  the  interactive  computer  graphics 
program  enabling  them  to  move  around  in  the  virtual  engineering  spaces.  Two  subject  can  be 
trained  together,  but  they  should  practice  and  be  tested  individually.  Table  1  shows  the  design  of 
the  evaluation. 
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Table  1 


VET-E  Evaluation  Plan 


Group  A 

Group  B 

Group  C 

Scenario  1  Training 

Scenario  1  Training 

Scenario  1  Training 

Photo  Visuals 

Photos  and  Case  Instruction  (Cl) 

Virtual  Environment  and  Cl 

Scenario  2  Test 

Scenario  2  Test 

Scenario  2  Test 

(Photos) 

(Photos) 

(Computer) 

In  addition  to  the  training,  each  group  will  be  given  a  short  demographic  questionnaire. 
Groups  B  and  C  will  be  given  a  questionnaire  on  the  case-based  approach.  Group  C  will  be  given 
a  questionnaire  on  the  usability  of  the  desktop  VE  program.  The  student  questionnaire  is  found  in 
Appendix  E.  Instructors  will  be  given  a  questionnaire  on  the  case-based  approach  and  the 
usability  of  the  desktop  VE  program.  The  instructor  questionnaire  is  found  in  Appendix  F. 

Since  the  virtual  environment  will  not  be  completed  until  February  1998,  Control  Group  B 
will  be  tested  for  the  January  class.  Group  C  for  the  May  class,  and  Group  A  for  the  July  class. 
Ideally,  some  subjects  would  be  tested  from  each  group  for  each  class.  However,  since  that  is  not 
possible  due  to  the  virtual  environment  completion  schedule,  demographics  for  each  subject  will 
be  examined  to  determine  that  the  groups  are  comparable  in  experience,  education,  and  training. 
If  they  are  not,  the  differences  will  be  controlled  statistically. 

F  amiliarization 

While  the  training  program  will  be  conducted  using  the  image-based  system,  it  would  still  be 
desirable  to  perform  some  evaluation  of  the  model-based  system.  The  model-based  system  could 
be  used  for  familiarization  of  the  engineering  spaces  and  it  could  be  evaluated  to  determine  if  this 
added  to  any  effect  of  the  image-based  system.  This  evaluation  could  take  place  if  there  are  a 
sufficient  number  of  subjects,  perhaps  an  additional  10,  to  add  this  component. 

For  the  group  trained  with  the  virtual  environment  on  the  computer,  a  familiarization  session 
using  a  model-based  computer  environment  simulation  of  the  engineering  spaces  could  be  used. 
Group  D  could  be  formed  and  trained  in  the  same  way  as  Group  C  with  the  addition  of  a 
familiarization  session  with  the  model-based  system  to  be  administered  to  the  image-based 
training.  This  training  would  allow  the  student  to  move  about  in  the  engineering  spaces  and 
locate  a  list  of  equipment.  Group  C  and  Group  D  would  be  compared  to  determine  if  this 
familiarization  session  increases  achievement  in  the  final  evaluation. 

For  the  familiarization  training,  the  instructor  will  show  the  student  how  to  move  about  in  the 
spaces  following  a  layout.  Then  the  instructor  will  demonstrate  how  to  find  several  items  on  a 
list.  The  instructor  will  watch  the  student  and  assist  as  necessary  for  a  few  items.  After  that, 
students  will  be  on  their  own  to  explore  the  spaces  and  find  the  equipment  on  the  list. 
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If  sufficient  subjects  are  not  available  to  include  Group  D,  the  model-based  system  could  be 
used  with  other  groups  of  subjects  who  are  now  required  to  pass  a  test  on  the  layout  of  a  ship. 
Groups  could  be  tested  with  and  without  the  aid  of  the  model-based  program  and  the  results  of 
the  test  could  be  compared  for  each  group.  While  this  would  show  a  possible  additional 
application  of  this  program,  it  is  not  part  of  the  objective  of  the  present  research  effort. 

Hypotheses 

1.  Group  C  will  perform  better  than  Groups  A  or  B.  It  is  expected  that  the  experimental 
treatment  with  both  the  computer  graphics  and  the  case-based  approach  will  be  superior. 

2.  Group  B  will  perform  better  than  Group  A.  Group  B  should  benefit  from  the  case-based 
approach,  but  will  not  have  the  interactive  computer  program  of  Group  C. 

3.  Group  A’s  performance  will  be  inferior  to  both  Group  B  and  Group  C.  Group  A  will  have 
neither  of  the  two  elements  of  Group  C,  nor  the  one  element  of  Group  B. 

Evaluation 

For  each  of  discrepancies  found,  the  subject  will  be  asked  the  following  questions. 

a.  Identify  the  discrepancy 

b.  Specify  what  action  to  take. 

c.  Prioritize  each  action  on  a  scale  of  1  to  5  indicating  when  the  action  should  be  taken: 

1.  Security  breach  or  CAS  REP. 

2.  Fix  immediately. 

3.  Put  on  the  8  o’clock  report — ^fix  this  week. 

4.  Place  discrepancy  on  the  CSMP  or  order  repair  parts. 

5.  Conduct  repairs  after  completing  higher  priorities. 

d.  Assess  the  situation.  What  are  the  implications  of  this  problem  for  the  situation  as  a 
whole. 

In  addition  to  the  quantitative  measures,  questionnaires  will  be  administered  to  each  subject 
to  collect  demographic  data  on  background,  education,  training,  and  experience.  In  addition,  a 
usability  questionnaire  will  be  administered  to  determine  the  subjective  experience  of  each 
subject  and  instructor  in  the  use  of  the  virtual  environment,  the  visual  pictures,  and  the  case- 
based  approach  to  instruction.  The  student  and  instructor  questionnaires  are  found  in  Appendices 
D  and  E,  respectively. 


15 


Analysis 


For  each  of  the  three  groups,  the  following  scores  will  be  collected  from  the  test 

1.  Identification  of  discrepancy  (DISCR).  This  is  a  measure  of  pattern  recognition.  There  are 
a  total  of  17  discrepancies  to  be  identified,  so  the  total  possible  DISCR  score  is  17.  There 
are  an  additional  six  nodes  to  be  examined  that  do  not  contain  any  discrepancies.  A  half¬ 
point  will  be  deducted  from  the  total  score  for  every  incorrect  discrepancy  identified, 
giving  a  DISCRA  score,  or  adjusted  discrepancy  score. 

2.  Corrective  action  (CORRA).  This  is  a  measure  of  management  skill.  The  answers  to  this 
question  will  be  rated  as  either  1  for  acceptable  or  2  for  good  insight  or  0  for  incorrect. 
Since  this  score  is  dependent  on  the  number  of  correct  discrepancies  identified,  it  will  be 
adjusted  for  the  DISCR  score  obtained.  The  CORRA  score  is  the  total  corrective  action 
score  divided  by  DISCR  multiplied  by  100  (to  remove  the  decimal). 

3.  Priority  (PRIOR).  This  is  another  measure  of  management  skill.  The  answers  to  this 
question  will  be  rated  as  either  1  for  acceptable  or  2  for  the  best  answer  or  0  for  incorrect. 
Since  this  measure  is  also  dependent  on  the  DISCR  score,  it  will  be  adjusted  accordingly. 
The  PRIOR  score  is  the  total  priority  score  divided  by  DISCR  multiplied  by  100. 

4.  Relation  to  other  systems  (SYSTM).  This  is  a  measure  of  system  knowledge.  The  answer 
to  this  question  will  be  rated  as  either  1  for  acceptable  or  2  for  the  best  answer  or  0  for 
incorrect.  Again  this  measure  is  dependent  on  the  DISCR  score  and  will  be  adjusted 
accordingly.  The  SYSTM  score  is  the  total  system  score  divided  by  DISCR  multiolied  bv 
100. 

Final  course  test  scores  (FINAL)  will  be  obtained  for  each  participant  in  the  study  to 
determine  if  the  interventions  had  any  effect  on  course  performance. 

A  multivariate  analysis  of  variance  (MANOVA)  with  three  independent  groups  (A,  B,  C)  and 
five  dependent  variable  (DISCR,  CORRA,  PRIOR,  SYSTM,  FINAL)  will  be  used.  Covariates 
such  as  years  of  experience  and  past  experience  can  be  controlled  in  this  design.  The  Wilks’  Test 
is  recommended  because  it  is  a  likelihood  ratio  test  and  has  good  properties  (Johnson,  1996). 

Pilot  Study 

First,  the  script  and  photos  were  tested  for  Group  B  on  a  Navy  Lieutenant  and  a  Senior  Chief 
to  ensure  the  instructions  were  clear,  to  determine  how  long  the  instruction  and  testing  would 
take,  and  to  determine  if  the  discrepancies  could  be  detected  in  the  photos.  Both  had  engineering 
and  damage  control  experience  but  neither  was  considered  an  expert.  Since  they  were  not 
subject  matter  experts,  we  did  not  suggest  changes  to  be  made  on  the  basis  of  their  input.  We 
did,  however,  make  changes  to  the  script  on  the  basis  of  their  testing. 

Second,  two  Lieutenants  who  were  engineers  were  tested;  one  officer  had  served  as  a 
Division  Officer  on  a  DDG51  and  the  other  had  18  years  experience  in  various  engineering 
capacities.  We  tested  the  script  on  both  officers,  and  then  had  one  of  them  complete  Scenario  1 
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and  the  other  Scenario  2.  After  they  completed  the  task,  we  examined  the  answer  sheets  to 
determine  if  they  had  failed  to  detect  any  of  the  discrepancies.  One  engineer  found  all  of  the 
discrepancies  and  the  other  missed  a  few.  In  some  cases,  the  discrepancies  that  were  missed 
were  seen  when  pointed  out,  and  in  others  the  discrepancies  were  considered  to  be  problematic. 
The  problematic  discrepancies  were  listed  and  recommended  for  adjustments  by  NAWCTSD 
after  other  experts  are  consulted,  probably  the  instructors  at  SWOS. 

Status  to  Date 

The  VET-E  project  was  transferred  in  its  final  year  to  NAWCTSD,  Orlando,  FL.  The 
transfer  is  the  result  of  the  NPRDC  Training  Department  function  being  transferred  to 
NAWCTSD  as  a  consequence  of  BRAC  95.  The  foregoing  materials  describe  the  background 
and  status  of  the  project  as  of  the  5*  of  December,  1997.  The  future  direction  and  development 
of  the  project  will  be  determined  by  NAWCTSD,  in  concert  with  the  membership  of  the  IPG. 
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Introduction 


The  Navy  Personnel  Research  and  Development  Center  (NPRDC)  has  been  directed  to 
develop  a  training  environment  that  will  assist  in  enabling  Naval  Engineering  Officers  to 
adequately  assess  the  material  condition  of  their  engineering  plants.  Instructional  Science  and 
Development  Inc.  (ISD)  has  done  extensive  work  in  shipboard  main  propulsion  engineering 
environments  and  is  familiar  with  the  unique  training  problems  that  arise  in  environments  that 
are  both  complex  and  hostile.  ISD  has  been  tasked  to  provide  NPRDC  with  two  detailed 
descriptions  of  material  assessment  scenarios,  to  be  modeled  in  a  virtual  environment  (VE) 
resembling  a  DDG51  class  ships  main  engine  room  (MER). 

ISD  has  been  tasked  to  (1)  evaluate  and  analyze  shipboard  material  inspection  data  and 
training  course  curriculum  materials  from  the  PEO  Course,  and  (2)  to  provide  NPRDC  with  two 
detailed  descriptions  of  material  assessment  scenarios.  These  scenarios  will  be  developed  and 
demonstrated  in  a  virtual  environment  (VE)  for  potential  inclusion  in  the  PEO  course  curriculum. 

From  a  training  standpoint  there  are  two  major  factors  that  determine  whether  discrepancies 
can  be  properly  identified  by  the  student,  they  are:  (1)  Does  the  student  have  the  appropriate 
program  knowledge  to  recognize  that  a  discrepancy  exists  and  to  identify  it?  (2)  Is  the  student 
familiar  enough  with  the  surrounds  to  be  able  to  focus  attention  and  recognize,  objects  that  have 
discrepancies  without  being  distracted  by  environmental  considerations?  If  the  student  has 
acquired  the  prerequisite  program  knowledge  needed  to  identify  the  discrepancies,  then  the  VE 
scenario  exercises  will  be  instrumental  in  familiarizing  them  with  the  environment.  In  addition, 
this  environment  provides  the  student  with  a  chance  visually  to  identify  discrepancies  in  a  more 
realistic  setting. 

Methodology 

The  data  analysis  was  conducted  in  two  phases.  First  the  Operational  Propulsion  Plant 
Examination  (OPPE)  inspection  data  was  analyzed.  Second  the  PEO  course  curriculum  and  class 
schedule  data  was  reviewed.  Using  the  findings  from  the  analysis,  specific  deficiencies  were 
identified  for  inclusion  in  the  scenarios.  The  deficiencies  were  further  broken  down  into 
subgroups  as  described  below.  Shipboard  walk-throughs  were  conducted  to  orchestrate  the 
inclusion  of  the  deficiencies  into  the  training  scenarios.  These  walk-throughs  provided  a 
sequence  for  displaying  equipment,  piping,  and  machinery  the  students  need  to  see  during  the  VE 
experience.  Consideration  was  also  given  to  the  sensory  perceptions  used  in  the  actual 
engineering  plant  environment,  as  well  as  comments  and  recommendations  on  their  usage  in  the 
virtual  environment. 

OPPE  Inspection  Results 

The  comprehensive  data  analysis  included  OPPE  inspection  reports  from  over  31  different 
ships  and  across  the  three  major  engineering  platforms:  steam,  gas  turbine,  and  diesel  propulsion. 
Specific  material  deficiencies  were  reviewed  and  deficiencies  that  were  general  and  found 
throughout  the  three  engineering  platforms  were  collected.  Data  found  to  occur  with  a  high 
degree  of  frequency  across  the  three  platforms  were  further  divided  into  six  subgroups.  Items  for 
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inclusion  in  the  scenarios  were  identified  from  these  subgroups.  The  subgroups  and  their 
individual  items  are  listed  below. 

•  Piping  and  Valves 

1 .  Missing  and  improper  valve  labeling 

2.  Poor  valve  maintenance 

3.  Improper  pipe  stenciling 

4.  Lagging  missing  and/or  tom 

5.  Flange  leaks 

6.  Short  studded  valves  and  flanges 

•  Damage  Control  Equipment 

1.  AFFF  stations 

2.  PKP  bottles  tags 

3.  Halon  stations 

•  General  Outfitting 

1.  Deck  plates 

2.  Unsafe  ladders 

3.  Improper  tag  out  procedures 

4.  Lock  wire  missing,  broken  etc. 

•  Instramentation 

1.  Gauges  out  of  calibration  and  improperly  installed 

2.  Thermometers  broken 

•  Flammable  Liquid  Systems 

1.  Lube  oil  strainers 

2.  Mechanical  seals  leaking 

3.  Flange  shields  soaked  and/or  missing 

4.  Lube  oil  sample  racks 

5.  Pooled  lube  oil  or  fuel  oil  on  foundations 

•  Machinery 

1.  Gas  Turbine  Module 

2.  Fuel  oil  service  pumps 

3.  Fuel  oil  and  lube  oil  purifiers 

4.  Lube  oil  service  pumps 

Most  of  the  machinery  listed  as  out  of  commission  (OOC)  during  the  time  of  the  OPPE 
inspections  was  removed  from  consideration.  This  machinery  varied  across  the  three  engineering 
platforms  and  would  serve  no  purpose  by  being  illustrated  in  the  virtual  environment.  However, 
one  piece  of  machinery,  should  appear  tagged  out  in  the  scenario  to  illustrate  improper  tag  out 
procedures. 
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PEO  Course  Curriculum  Review 


The  training  objectives  and  the  curriculum  schedule  from  the  Prospective  Engineering 
Officer  (PEO)  course  were  reviewed.  Of  the  72  class  periods  reflected  over  the  two  week  course 
schedule,  31  of  these  periods  contain  some  sort  of  material  assessment  activities.  These  material 
assessment  activities  include  assessing  safety,  damage  control  equipment,  cold  checks,  and  hot 
checks.  By  matching  the  results  from  the  data  analysis  of  shipboard  inspections  with  the  topics 
taught  in  the  PEO  course,  specific  areas  could  be  identified. 

A  significant  number  of  material  assessment  lessons  came  from  the  Engineering  Operational 
Sequencing  System  (EOSS)  procedure  Master  Light  Off  Checklist  (MLOC),  a  Prelightoff 
Checklist.  Items  addressed  in  the  MLOC  procedure  to  be  included  in  the  scenarios  are  listed 
below  as  they  appear  in  the  EOSS  procedure.  It  should  be  noted  that  extensive  knowledge  of  a 
number  of  engineering  programs,  both  in  material  and  maintenance  areas,  is  needed  to  comply 
with  some  of  the  listed  items. 

MLOC  Items 

•  Ensure  bilge,  drip  pans,  and  decks  are  free  of  oil. 

•  Ensure  all  flange  shields  and  strainer  spray  covers  are  in  place  and  properly  attached. 

•  Ensure  all  damage  control  equipment  is  in  place  and  ready  for  use  (weight,  seals,  charged, 
etc.) 

•  Ensure  all  valve  hand  wheels  are  installed,  properly  attached,  and  remote  operators  are 
connected  and  operational. 

•  Ensure  all  gauges  are  in  place,  properly  mounted  and  not  overdue  for  calibration. 

•  Ensure  all  security  locks,  lock  wire  seals,  and  locking  devices  are  in  place  and  no 
indication  of  being  tampered  with. 

•  Inspect  for  and  remove  all  fire  hazards  from  engineering  spaces.  Stow  all  equipment  not 
in  use  (missile  hazards). 

Shipboard  Environment  Observations  and  Findings 

A  visit  was  made  to  the  USS  John  Paul  Jones  DDG-53.  The  #2  main  engine  room  was  toured 
and  evaluated  for  the  development  of  the  two  scenarios.  The  engine  room  has  three  levels:  1st 
platform,  2nd  platform,  and  hold  level.  A  walk  through  of  each  level  was  conducted  and  various 
deficiency  items  were  identified.  Special  attention  was  paid  to  the  noise  levels,  temperature 

changes  in  various  areas,  and  the  visual  accessibility  of  frequently  noted  discrepancies  listed 
above. 


In  general,  the  actual  engineering  environment  overloads  the  senses.  Visually  there  are 
objects  in  close  proximity  to  one’s  body  from  overhead,  under  foot  and  on  either  side.  This 
causes  the  space  personnel  to  constantly  watch  the  overhead  for  possible  objects  hanging  down 
that  could  cause  injury  to  their  heads.  The  deck  surface  varies  from  deck  grating,  which  is  open 
so  that  the  lower  level  can  be  seen  from  above,  to  deck  plating,  which  is  closed.  This  variance  in 
deck  surface  causes  the  space  personnel  to  always  be  conscious  of  their  footing.  Numerous 
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objects  have  to  be  avoided  while  walking  around  the  space.  Personnel  have  to  maneuver  around 
the  space  rather  than  simply  walking  through. 

The  surfaces  of  various  machinery,  equipment  and  piping  vary  greatly  in  surface  temperature, 
which  causes  the  space  temperature  to  fluctuate.  Walking  under  a  vent  causes  personnel  to  feel 
cool  air  blowing  on  them,  while  walking  close  to  an  operating  air  compressor  or  similar  piece  of 
equipment  gives  off  radiant  heat  that  can  be  felt.  Usual  odorant  inputs  include  fuel  oil,  lube  oil, 
and  fire.  The  decibel  (Db)  levels  often  exceed  the  recommended  ranges  and  hearing  protection  is 
required  when  entering  a  space.  The  sounds  consist  of  equipment  operating  sounds,  various 
communications  over  the  loud  speaker  systems,  and  other  metallic  noises. 

These  combinations  of  sensory  inputs  hamper  personnel  in  identification  of  various 
discrepancies.  The  senses  of  the  student  should  be  exercised  as  much  as  possible  during  the 
virtual  experience.  The  virtual  environment  should  aid  in  familiarizing  the  student  with  the 
environment  in  which  they  will  be  operating.  Some  general  observations  are  listed  below 
concerning  sensory  inputs  the  students  will  need  to  experience  during  the  scenarios.. 

Sound 

Engine  room  sounds  need  to  be  represented  in  the  virtual  environment  and  the  student  should 
wear  ear  plug  like  devices  that  cause  the  same  uncomfortable  feeling  as  ear  plugs  do.  Disposable 
ear  plugs  are  required  for  all  personnel  going  into  a  noise  hazard  area  such  as  the  main  engine 
room.  The  noise  should  be  as  realistic  as  possible  and  noise  levels  should  vary  based  upon  the 
proximity  of  the  student  to  the  machinery  or  source.  The  specific  Db  levels  of  the  actual 
environment  should  be  met.  The  physical  discomfort  of  the  noise  coupled  with  the  discomfort  of 
the  ear  plugs  will  add  to  the  realism  of  the  sensory  inputs. 

Visual  Effects 

Most  of  the  material  deficiencies  in  the  two  scenarios  fall  within  the  field  of  vision  ranging 
between  five  and  seven  feet  in  height.  As  the  student  walks  through  the  engine  room  the 
discrepancies  are  either  at  eye  level,  slightly  above  or  slightly  below,  but  most  can  be  seen 
without  bending  down.  For  looking  at  pump  foundations  and  other  discrepancies  where  bending 
down  is  required,  a  zooming  feature  should  be  provided.  This  will  enable  the  student  to  take  a 
closer  look  at  various  equipment  and  provide  a  way  of  reading  information  on  various  tags, 
labels,  and  gauges.  Some  tags  and  labels  will  need  to  be  finely  detailed  so  that  the  student  can 
walk  up  to  a  gauge  and  read  not  only  the  graduations  on  the  gauge  but  also  the  small  lettering  on 
the  calibration  sticker  affixed  to  the  face  of  the  gauge.  The  student  should  also  be  able  to  view 
the  environment  with  the  normal  range  of  sight,  ranging  from  far  left  to  far  right,  or 
approximately  180  degrees.  Generally  the  closeness  of  the  environment  should  be  duplicated. 
The  height  of  overhead  objects  should  be  duplicated  giving  the  effect  of  hanging  close  to  the 
student,s  head.  Similarly  the  deck  grating  placement  and  surrounding  machinery  should  be 
placed  so  the  student  sees  the  objects  in  the  same  proportion  as  onboard  ship.  On  the  hold  level 
of  the  engine  room  the  student  is  required  to  step  up  and  down  a  few  steps.  The  student  should 
see  the  effects  of  stepping  up  or  stepping  down  as  the  case  may  be.  Finally  during  the  time  the 
students  are  in  the  virtual  environment  their  vision  should  be  restricted  to  the  virtual 
environment.  No  outside  stimulus  should  be  present. 


A-5 


Maneuverability 

For  the  students  to  maneuver  throughout  the  space  and  between  the  levels  they  will  need  a 
device  that  gives  them  freedom  of  movement.  They  should,  however,  be  required  to  maneuver 
around  all  machinery  and  other  objects  in  their  way.  The  students  should  be  given  some  negative 
indication  when  they  bump  into  an  object  thereby  forcing  them  to  constantly  pay  attention  to 
where  they  are  going.  This  can  be  accomplished  using  a  joystick,  mouse  or  some  similar  device. 
The  students  should  be  able  to  move  ahead,  backward,  and  make  a  360  degrees  rotation  from 
either  side,  without  needing  to  move  ahead. 

Scenario  Scoring 

The  scenarios  need  to  be  scored  to  monitor  the  students’  progress  through  the  course.  There 
should  be  a  device  that  enables  the  students  to  mark  the  discrepancies  they  find  during  the 
scenario.  Once  an  object  is  marked  it  should  indicate  to  the  student  that  it  has  been  previously 
marked.  This  can  be  accomplished  by  indicating  it  visually  or  by  producing  a  sound  that  tells  the 
student  that  it  already  has  been  marked. 


A-6 


Scenario  #1 


The  starting  point  of  the  scenario  places  the  student  standing  on  the  1st  platform  of  #2  main 
engine  room.  The  down  ladder  is  directly  behind  them  and  the  student  is  facing  the  AFFF  hose 
reel  which  is  slightly  to  the  right. 

AFFF  Station  #1  The  following  discrepancies  exist:  (1)  The  hose  is  tangled  and  not  properly 
wound  around  the  reel.  (2)  The  APTF  inlet  valve  has  safety  wire  hanging 
and  not  connected  to  valve  properly. 

After  inspection  of  the  AFFF  station,  the  student  then  moves  to  the  down  ladder  between  #2 
GTG  and  #2A  GTM,  and  goes  to  the  2nd  platform.  On  the  2nd  platform  the  student  turns  to  the 
left  and  passes  between  the  GTM  module  and  the  ladder. 

2A  GTM  Module  The  module  door  is  closed  and  the  lock  is  missing  from  the  door.  Details 
of  the  door  and  the  lock  hinge  have  to  be  shown,  including  any  warning 
signs  that  are  present. 

After  walking  pass  the  module,  the  student  moves  to  the  area  directly  behind  #2  GTG.  As  the 
student  looks  straight  ahead  he/she  sees  the  rear  of  the  GTG,  the  fire  station  to  the  left  and  the 
halon  cylinders  to  the  right.  As  the  student  stands  looking  at  the  stator  of  the  GTG,  the  following 
discrepancies  can  be  seen.  The  student  should  be  able  to  move  close  to  each  of  the  three  objects 
for  inspection. 

#2  GTG  (rear)  The  stator  oil  piping  has  two  flange  shields  missing,  and  lube  oil  is  pooled 

on  the  foundation. 

Fire  Station  #1  The  fire  station  has  obvious  standing  water  in  its  hose  coming  from  the 

valve  to  the  first  hose  loop.  This  is  indicated  by  the  hose  being  swollen  at 
this  point. 

Halon  Cylinders  #1  The  pressure  gauges  on  the  halon  bottles  indicate  zero. 

Moving  to  the  right  the  student  is  allowed  to  walk  past  the  halon  cylinders.  Making  a  left  turn 
and  moving  straight  ahead  the  student  moves  down  the  side  of  the  GTG  to  the  module  door. 

GTG  #2  Module  The  module  door  is  closed  and  the  lock  is  missing  from  the  door.  Details 
of  the  door  and  the  lock  hinge  have  to  be  shown,  including  any  warning 
signs  that  are  present. 

The  student  can  move  forward  pass  the  GTG  module  door  to  the  fire  station,  but  no  further. 

Fire  Station  #2  The  nozzle  has  small  drips  of  saltwater  coming  from  the  tip,  causing  the 
deck  directly  below  it  to  have  salt  caked  on  it. 

The  student  is  then  allowed  to  walk  back  down  the  path  just  taken,  allowed  to  move  to  the 
right  between  the  dry  air  system  dehydrator  #4  and  SSAS  air  dehydrator  #2  or  to  the  left  past  the 
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halon  cylinders  and  the  start  air  reducing  station.  No  further  discrepancies  can  be  noted  if  the 
student  moves  back  towards  the  area  s/he  came  from,  or  if  s/he  decide  to  turn  left  and  go  past  the 
halon  cylinders.  If  the  student  decides  to  turn  right,  once  in  front  of  #2  LPAC,  the  student  walks 
down  past  the  air  compressors  and  dehydrators.  The  noise  levels  in  this  area  should  increase,  as 
appropriate,  with  #3  LPAC  operating.  The  student  can  then  continue  to  walk  past  #2  HPAC  with 
#2  HP  dehydrator  on  the  opposite  side. 

#2  HP  Air  Dehydrator  There  are  three  gauges  on  this  dehydrator.  The  calibration  stickers 

should  be  missing  and  the  red  line  pointers  on  the  gauges  should 
be  pointing  to  zero. 

Heat  Stress  Thermoineter  Place  a  heat  stress  thermometer  in  the  overhead  hanging  from  the 

electrical  cables.  The  thermometer  glass  column  should  appear  to 
be  broken. 

The  scenario  should  end  when  all  discrepancies  are  found  and  identified  or  after  30  minutes 
of  operation. 
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Scenario  #2 


The  starting  point  of  this  scenario  places  the  student  standing  on  the  hold  platform  of  #2  main 
engine  room  at  the  foot  of  the  down  ladder.  With  the  ladder  directly  behind,  the  student  walks 
straight  ahead  as  he/she  is  facing  the  lube  oil  service  pumps.  The  fuel  oil  service  filter/separators 
are  positioned  to  the  right. 


Fuel  Service  Filter/  The  gauges  on  the  gauge  board  for  the  filter/separators  havecalibration 
Senaration  2A  &2R  stickers  missing,  and  half  of  the  red  line  pointers  are  at  zero.  The  other  half 
are  at  the  proper  locations. 


Fire  station  #2 


The  fire  station  has  no  discrepancies. 


PKP  Bottle#! 


The  PKP  bottle  via  the  AFFF  station  to  the  right  of  the  student  has  a  tag 
tom  and  oil  soaked. 


AFFF  Station  #2  The  following  discrepancies  exist  on  the  AFFF  station  to  the  student’s 
right:  (1)  The  hose  is  tangled  and  not  properly  wound  around  the  reel.  (2) 
The  AFFF  inlet  valve  is  missing  the  required  safety  wire  that  should  be 
connected  to  valve. 


After  inspection  of  the  AFFF  station,  the  student  then  moves  to  the  lube  oil  service  pumps. 
While  standing  in  front  of  2B  lube  oil  service  pump  and  looking  at  the  two  vertical  dischzirge 
valves,  he/she  sees  the  following  discrepancies: 

2B  Lube  Oil  The  flange  shields  on  the  lower  valve  are  oil  soaked  and  there  is  pooled  oil 

Service  Pump  on  the  flange.  The  handwheel  is  missing  a  proper  valve  label  and  there  is 

lube  oil  pooled  on  the  foundation  of  the  service  pump. 


Walking  past  both  of  the  lube  oil  service  pumps  and  moving  along  straight  ahead,  on  the 
student’s  left  is  a  duplex  lube  oil  strainer. 


Duplex  Lube  Oil  The  strainer  has  pooled  lube  oil  standing  on  the  foundation  and  an  oily  rag 
Strainer  is  balled  up  in  the  drip  pan. 


Heat  Stress 
Thermometer 


Place  a  heat  stress  thermometer  in  the  overhead  hanging  from  a  pipe 
directly  above  the  service  pump.  The  thermometer  should  show  the 
mercury  at  the  top  of  the  glass  column. 


The  student  walks  to  the  duplex  saltwater  strainer  and  makes  a  right  turn.  The  student  now 
faces  the  port  shaft.  He/she  sees  the  shaft  turning  (the  shaft  is  painted  with  barber  pole  stripes). 
At  this  point  there  is  a  single  step  going  down  that  allows  personnel  to  step  under  the  shaft  and  a 
single  step  going  up  on  the  other  side  of  the  shaft.  The  student  should  be  able  to  see  the  steps  and 
be  able  to  walk  forward  and  see  the  effects  of  stepping  down  and  stepping  up  while  passing 
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under  the  shaft.  Once  the  student  has  passed  under  the  shaft  and  stepped  up  on  the  other  side, 
he/she  is  allowed  to  go  to  the  fire  station  and  no  further. 


Fluorescent  Light  #1  A  fluorescent  light  in  the  overhead  in  front  of  the  fire  station  is  flickering, 

indicating  a  tube  getting  ready  to  go  out. 


Fire  station  #3 


The  fire  station  has  salt  caked  on  the  top  of  the  hose  and  the  indicator 
loop  is  full  of  standing  water  (refer  to  fire  station  #1). 


The  student  returns  under  the  shaft  the  same  way  he/she  came.  Walking  past  the  duplex  oil 

strainer  on  the  right,  and  fire-pump  #4  on  the  left,  the  student  sees  the  fuel  service  heater 
controller  on  their  left 


Fuel  Service  Heater  This  controller  is  tagged  out  with  a  red  tag  hanging  on  the  controller. 
Controller  Walking  close  to  the  controller  the  student  sees  that  the  date  and  time  is 

missing  from  the  tag. 

Continuing  to  walk  forward  the  student  walks  past  the  AHFF  hose  reel  and  stops  parallel  to 
the  ladder  he/she  came  down.  Looking  to  the  left  they  see  2A  fuel  oil  service  pump. 

2A  Fuel  Oil  Service  The  service  pump  is  missing  a  safety  guard  around  the  shaft,  and  fuel  oil 

is  pooled  on  the  foundation  of  the  pump.  Additionally  one  flange  shield  is 
missing. 

The  scenario  should  end  when  all  discrepancies  are  found  and  identified  or  after  45  minutes 
of  operation. 
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Background 


The  Navy  Personnel  Research  and  Development  Center  (NPRDC)  has  been  directed  to 
develop  a  training  environment  that  will  assist  in  enabling  Naval  Engineering  Officers  to 
adequately  assess  the  material  condition  of  engineering  plants.  The  Virtual  Environment 
Training  for  Engineering  (VET-E)  Project  was  developed  to  reinforce  classroom  training  with  an 
environment  that  augments  the  training  communities  efforts  in  providing  material  assessment 
training.  Instructional  Science  and  Development  Inc.  (ISD)  was  tasked  to  provide  NPRDC  with 
descriptions  of  two  detailed  material  assessment  scenarios  (Contract  N66001-91-D-9502,  Order 
7J21,  March  1996)  to  be  modeled  in  a  virtual  environment  (VE)  resembling  a  DDG51  class  main 

engine  room  (MER).  The  scenarios  were  developed  for  potential  inclusion  in  the  PEO  course 
curriculum. 

To  develop  effective  training  scenarios  ISD  employed  a  methodology  which  included 
conducing  a  data  analysis  of  (a)  the  Operational  Propulsion  Plant  Examination  (OPPE) 
inspection  data  and  (b)  the  PEO  course  materials.  OPPE  inspection  data  from  over  31  different 
ships  were  reviewed,  along  with  the  PEO  course  curriculum  arid  materials  covering  72  hours  of 
classroom  instruction  over  the  two  week  period  of  the  course.  Using  the  findings  from  the 
analysis,  specific  deficiencies  were  identified  for  inclusion  in  the  scenarios.  The  material 
deficiencies  were  categorized  into  subgroups  which  were  included  in  the  report. 

In  addition  to  the  data  analysis,  shipboard  walk-throughs  were  conducted  on  board  the  USS 
John  Paul  Jones  DDG-53,  to  orchestrate  the  inclusion  of  the  deficiencies  into  the  training 
scenarios.  These  walk-throughs  provided  a  sequence  for  displaying  equipment,  piping,  and 
machinery  the  students  need  to  see  during  the  VET.  Consideration  was  also  given  to  the  sensory 
perceptions  used  in  the  actual  engineering  plant  environment  by  personnel,  as  well  as  comments 
and  recommendations  made  by  ships  company  about  their  usage  in  the  virtual  environment. 

Introduction 


On  24  October  1996,  roughly  seven  months  after  the  submission  of  the  original  training 
scenario  design  plan  report,  a  meeting  was  held  at  Surface  Warfare  Officer  School  (SWOS) 
Engineering  Department  Head  School,  Newport,  RI.  Representatives  from  NPRDC  and  the 
engineering  department  head  school  were  present.  The  purpose  of  the  meeting  was  to  clarify 
objectives  for  the  VET-E  project  and  to  review  and  discuss  the  scenario  content  for  usage  in  the 
project.  It  was  determined  that  the  current  scenarios  were  too  simple  to  stimulate  the  critical 
thinking  required  at  the  department  head  level.  It  was  determined  that  modification  of  the 
existing  scenarios  is  required  to  make  them  more  cognitively  challenging. 

To  fully  understand  the  concerns  and  desires  of  the  SWOS  principals,  ISD  has  communicated 
directly  with  SWOS  personnel  concerning  the  scenario  modifications.  Two  major  concerns  were 
aired  during  the  conversation.  The  first  concern  was  the  cognitive  recognition  of  material 
discrepancies  based  on  multiple  indicators,  and  the  second  concern  was  the  development  of  time- 
based  scenarios  with  progressive  deterioration  of  material  conditions.  The  former  requires  the 
student  to  correlate  data  from  multiple  sources  and  to  recall  that  information  during  the  scenario 
walk-throughs.  The  latter  requires  that  progressive  scenarios  be  developed  that,  over  time,  cause 
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material  discrepancies  to  develop  into  dangerous  situations.  This  discussion  and  the  ideas 
presented  were  passed  along  to  the  NPRDC  representative  for  future  consideration. 

This  report  modifies  the  original  training  scenarios  by  adding  additional  discrepancies  to  the 
equipment  presented  in  the  original  scenarios.  The  additional  discrepancies  are  indicated  in  this 
report  by  bolded  print.  It  is  estimated  that  the  additional  discrepancies  will  increase  the  length  of 
each  scenario  by  approximately  10  minutes  per  scenario. 
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Scenario  No.  1 


In  this  scenario  the  student  is  standing  on  the  1st  platform  of  No.2  Main  Engine  Room 
(MER).  The  student  will  travel  from  the  1st  platform  to  the  2nd  platform.  The  down  ladder  is 
directly  behind  them  and  the  student  is  facing  the  AFFF  hose  reel  which  is  slightly  to  the  right. 
The  student  moves  to  the  AFFF  station. 

AFFF  Station  #1 

The  following  discrepancies  exist:  (1)  The  hose  is  tangled  and  not  properly  wound  around  the 
reel.  (2)  The  AFFF  inlet  valve  has  safety  wire  hanging  and  not  connected  to  valve  properly. 

(3)  The  nozzle  is  visible  and  appears  to  be  clogged  with  debris. 

Following  the  inspection  of  the  AFFF  station,  the  student  then  moves  to  the  down  ladder 
between  No.2  GTG  and  No.2A  GTM.  As  the  student  approaches  the  down  ladder  there  is  a  heat 
stress  thermometer  hanging  from  the  overhead. 

Heat  Stress  Thermometer 

The  thermometer  should  appear  to  indicate  a  temperature  in  excess  of  100  degrees 
Fahrenheit. 

The  student  then  travels  down  the  ladder  to  the  2nd  platform.  On  the  2nd  platform  the  student 
turns  to  the  left  and  passes  between  the  #2A  GTM  module  and  the  ladder. 

2A  GTM  Module 

The  module  door  is  closed  and  the  lock  is  missing  from  the  door.  Details  of  the  door  and  the 
lock  hinge  have  to  be  shown,  including  any  warning  signs  that  are  present. 

After  walking  pass  the  module,  the  student  moves  to  the  area  directly  behind  No.  2  GTG.  As  the 
student  looks  straight  ahead  they  see  the  rear  of  the  GTG,  the  fire  station  to  the  left  and  the  halon 
cylinders  to  the  right.  As  the  student  stands  looking  at  the  stator  of  the  GTG,  the  following 
discrepancies  can  be  seen.  The  student  should  be  able  to  move  close  to  each  of  the  three  objects 
for  inspection. 

No.2  GTG  (rear) 

The  stator  oil  piping  has  two  flange  shields  missing.  On  the  two  flanges  missing  flange 
shields,  on  one  flange  there  is  a  missing  stud  and  the  other  flange  has  a  short  stud,  n 
addition  lube  oil  is  pooled  on  the  foundation.  One  flange  shield  is  loose  and  improperly 
attached. 


B-4 


Fire  station  #1 


The  fire  station  has  obvious  standing  water  in  its  hose  coming  from  the  valve  to  the  first  hose 
loop.  This  is  indicated  by  the  hose  being  swollen  at  this  point.  This  causes  the  loop  to  lie  on 
the  deck.  The  adjustable  spanner  wrench  is  missing  from  its  holder. 

Halon  Cylinders  #1 

The  pressure  gauges  on  the  halon  bottles  indicate  zero. 

Moving  to  the  right  the  student  is  allowed  to  walk  past  the  halon  cylinders.  Making  a  left  turn 
and  moving  straight  ahead  the  student  moves  down  the  side  of  the  GTG  to  the  module  door. 

No.2  GTG  Module 

The  module  door  is  closed  and  the  lock  is  missing  from  the  door.  Details  of  the  door  and  the 
lock  hinge  have  to  be  shown,  including  any  warning  signs  that  are  present. 

The  student  then  moves  forward  pass  the  GTG  module  door  to  the  fire  station. 

Fire  Station  #2 

The  nozzle  has  small  drips  of  saltwater  coming  from  the  tip,  causing  the  deck  directly  below 
it  to  have  salt  caked  on  it.  The  wye  gate  on  the  quick  cleaning  strainer  is  closed. 

The  student  is  then  allowed  to  walk  back  down  the  path  just  taken;  allowed  to  move  to  the  right 
between  the  dry  air  system  dehydrator  No.4  and  SS AS  air  dehydrator  No.  2  or;  allowed  to  move 
to  the  left  past  the  halon  cylinders  and  up  to  the  start  air  reducing  station.  No  further 
discrepancies  can  be  noted  if  the  student  moves  back  towards  the  area  they  came  from.  If  they 
decide  to  turn  left  and  go  past  the  halon  cylinders,  the  student  can  see  that  the  start  air  reducing 
manifold  is  tagged  out. 

Start  Air  Reducing  Manifold 

Danger  tags  are  hanging  from  the  start  air  reducing  manifold.  On  the  deck  is  a  danger 
tag  that  is  adrift  in  front  of  the  manifold. 

If  the  student  decides  to  turn  right  between  the  dry  air  system  dehydrator  No.4,  once  they  are  in 
front  of  No.  2  LPAC,  the  student  walks  down  past  the  air  compressors  and  dehydrators.  The 
noise  levels  in  this  area  should  increase,  as  appropriate,  with  #3  LPAC  operating.  The  student 
can  then  continue  to  walk  past  No.  2  HPAC  with  No.  2  HP  dehydrator  on  the  opposite  side. 

No.  2  HPAC  Seawater  Strainer 

The  seawater  strainer  has  caked  salt  around  the  gasket  seal  indicating  a  leak.  A  small 
amount  of  seawater  is  seen  leaking  on  the  deck  plates. 
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No.  2  HP  Air  Dehydrator 

There  are  three  gauges  on  this  dehydrator.  The  calibration  stickers  should  be  missing  and  the 
red  line  pointers  on  the  gauges  should  be  pointing  to  zero.  One  gage  has  a  broken  face 
cover. 

Heat  Stress  Thermometer 


Place  a  heat  stress  thermometer  in  the  overhead  hanging  from  the  electrical  cables.  The 
thermometer  glass  column  should  appear  to  be  broken. 

HPAC  Controller 

As  the  student  passes  the  HPAC  controller  they  see  that  a  caution  tag  is  hanging  from 
the  controller.  If  they  inspect  the  tag  closer  they  see  that  there  is  no  signature  or  date  on 
the  tag. 


The  scenario  should  end  when  all  discrepancies  are  found  and  identified  or  after  45  minutes  of 
operation. 
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Scenario  No.  2 


The  starting  point  of  this  scenario  places  the  student  standing  on  the  hold  platform  of  #2  main 
engine  room  at  the  foot  of  the  down  ladder.  With  the  ladder  directly  behind,  the  student  walks 
straight  ahead  as  he/she  is  facing  the  lube  oil  service  pumps.  The  fuel  oil  service  filter/separators 
are  positioned  to  the  right. 

Fuel  Service  Filter/Separation  2A  &  2B 

The  gauges  on  the  gauge  board  for  the  filter/separators  have  calibration  stickers  missing,  and 
half  of  the  red  line  pointers  are  at  zero.  The  other  half  are  at  the  proper  locations. 

Fire  station  #2 

1.  The  adjustable  spanner  wrench  is  missing  from  its  holder. 

2.  The  multi-purpose  nozzle  is  missing  form  the  hose  end. 

PKP  Bottle  1 

The  PKP  bottle  via  the  AFFF  station  to  the  right  of  the  student  has  a  tag  tom  and  oil  soaked. 

AFFF  Station  #2 

The  following  discrepancies  exist  on  the  AFFF  station  to  the  student’s  right: 

1 .  The  hose  is  tangled  and  not  properly  wound  around  the  reel. 

2.  The  AFFF  inlet  valve  is  missing  the  required  safety  wire  that  should  be  connected  to 
valve. 

Following  the  inspection  of  the  AFFF  station,  the  student  moves  down  to  the  seawater  cooling 
pump. 

Seawater  Cooling  Pump  No.  4 

The  cooling  pump  is  operating  and  the  coupling  guard  is  missing. 

The  student  then  moves  to  the  lube  oil  service  pumps.  While  standing  in  front  of  2B  lube  oil 
service  pump  and  looking  at  the  two  vertical  discharge  valves,  they  see  the  following 
discrepancies: 

2B  Lube  Oil  Service  Pump 

1 .  The  flange  shields  on  the  lower  valve  are  oil  soaked  and  there  is  pooled  oil  on  the  flange. 

2.  The  handwheel  is  missing  a  proper  valve  label  and  there  is  lube  oil  pooled  on  the 
foundation  of  the  service  pump. 
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2A  Lube  Oil  Service  Pump 

The  remote  operator  on  the  suction  valve  of  2A  LOSP  is  disconnected  and  lying  on  the 
valve  wheel. 

Walking  past  both  of  the  lube  oil  service  pumps  and  moving  along  straight  ahead,  on  the 
student’s  left  is  a  duplex  lube  oil  strainer.  If  the  student  turns  around  they  will  see  No.  4  Fire 
Pump 

No.  4  Fire  Pump 

The  automatic  bus  transfer  for  the  fire  pump  has  been  tagged  out  and  the  danger  tag 
hanging  from  the  controller  is  torn  and  half  of  the  tag  is  missing. 

Duplex  Lube  Oil  Strainer 


The  strainer  has  pooled  lube  oil  standing  on  the  foundation  and  an  oily  rag  is  balled  up  in  the 
drip  pan  and  the  strainer  cover  is  off  the  strainer. 

Heat  Stress  Thermometer 

Place  a  heat  stress  thermometer  in  the  overhead  hanging  from  a  pipe  directly  above  the 
service  pump.  The  thermometer  should  show  the  mercury  at  the  top  of  the  glass  column. 

The  student  walks  past  the  duplex  saltwater  strainer  and  makes  a  right  turn.  On  the  right  is  the 
lube  oil  purifier. 

No.  2  Lube  Oil  Purifier 

Two  flange  shields  are  improperly  attached  and  oil  soaked. 

After  identifying  the  discrepancies  on  the  lube  oil  purifier  the  student  faces  the  port  shaft.  They 
see  the  shaft  turning  (the  shaft  is  painted  with  barber  pole  stripes). 

Fluorescent  Light  #1 

A  fluorescent  light  in  the  overhead  on  top  of  the  shaft  is  flickering,  indicating  a  tube  getting 
ready  to  go  out.  This  flickering  can  cause  the  shaft  to  appear  to  be  stopped  and  is  a  safety 
hazard. 


At  this  point  there  is  a  single  step  going  down  that  allows  personnel  to  step  under  the  shaft  and  a 
single  step  going  up  on  the  other  side  of  the  shaft.  The  student  should  be  able  to  see  the  steps  and 
be  able  to  walk  forward  and  see  the  affects  of  stepping  down  and  stepping  up  while  passing 
under  the  shaft.  Once  the  student  has  passed  under  the  shaft  and  stepped  up  on  the  other  side, 
they  are  allowed  to  go  to  the  fire  station  and  no  further. 
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Fire  station  #3 


The  fire  station  has  salt  caked  on  the  top  of  the  hose  and  the  indicator  loop  is  full  of  standing 
water  (refer  to  fire  station  #1). 

The  student  returns  under  the  shaft  the  same  way  they  came.  Walking  past  the  duplex  oil  strainer 
on  the  right,  and  fire-pump  No.  4  on  the  left,  the  student  sees  the  fuel  service  heater  controller  on 
their  left 

Fuel  Service  Heater  Controller 

This  controller  is  tagged  out  with  a  red  tag  hanging  on  the  controller.  Walking  close  to  the 
controller  the  student  sees  that  the  date  and  time  is  missing  from  the  tag. 

Continuing  to  walk  forward  the  student  walks  past  the  AFFF  hose  reel  and  stops  parallel  to  the 
ladder  they  came  down.  Looking  to  the  left  they  see  2A  fuel  oil  service  pump. 

2A  Fuel  Oil  Service  Pump 

The  following  discrepancies  are  found  on  the  service  pump: 

1.  The  service  pump  is  missing  a  safety  guard  around  the  shaft 

2.  Fuel  oil  is  pooled  on  the  foundation  of  the  pump. 

3.  One  flange  shield  is  missing. 

4.  The  remote  operator  is  disconnected  on  the  pump  suction  valve. 

The  scenario  should  end  when  all  discrepancies  are  found  and  identified  or  after  55  minutes  of 
operation. 
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Appendix  C 

Case-Based  Instructional  Scripts 
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Control  Group  A  (Visuals  Only) 
Scenario  1 


Materials 

This  group  will  be  given  pictures  of  the  engineering  spaces  as  well  as  the  accompanying 
documents. 

Method 

Students  will  be  given  materials  and  general  written  instructions  and  will  proceed  on  their  own. 

Script 

We  are  looking  into  optimal  ways  of  teaching  material  inspection  to  Engineering  Officers.  You 
have  been  selected  to  try  out  some  new  visual  materials  that  have  been  developed  from  pictures 
taken  aboard  ship.  First  we  will  give  you  some  pictures  to  examine  for  45  minutes.  Then  we  will 
give  you  a  new  set  of  similar  pictures  and  ask  you  some  questions. 

Here  are  some  pictures  of  the  engineering  spaces  on  an  Arleigh  Burke  class  ship  where  you  are 
the  CHENG.  You  are  in  port  and  getting  ready  to  do  a  walk  through  your  spaces  to  assess  their 
material  readiness.  You  have  wheelbook  and  logs  available  to  you. 

Ordinarily  we  would  proceed  though  the  spaces  in  a  particular  order.  However,  due  to  time 
limits,  for  this  demonstration  we  have  selected  certain  locations  (nodes)  for  you  to  examine. 

Examine  the  following  areas  for  discrepancies.  Not  all  areas  will  have  discrepancies. 

-  Platform  1,  Nodes  1, 4, 7, 14, 23. 

--  Platform  2,  Nodes  4,  8,  14, 15,  6,  13. 

-  Hold,  Nodes  2, 4,  5,  12, 15,  16. 

For  each  discrepancy  you  find: 

1.  Consider  what  actions  you  would  take  to  correct  it. 

2.  Consider  how  you  would  prioritize  your  action  on  a  scale  of  1  to  5,  using  the  following 
categories: 

1 .  Security  breach  or  CAS  REP. 

2.  Fix  immediately. 

3.  Put  on  the  8  o’clock  report/  fix  this  week. 
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4.  Place  discrepancy  on  the  CSMP  or  order  repair  parts. 

5.  Conduct  repairs  or  correct  after  completing  higher  priorities. 

3.  If  applicable,  consider  how  this  discrepancy  affects  the  functioning  of  other  equipment  or 
systems  aboard  ship. 

4.  Consider,  what  information  led  you  to  this  situation  assessment  and  these  actions. 

5.  Consider  potential  errors  that  less  experienced  engineers  might  make. 

You  will  not  have  to  write  your  answers  to  these  questions.  You  many  consult  any  information 
you  wish. 

You  will  have  one  hour  for  this  part. 
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Control  Group  A  (Visuals  Only) 
Scenario  2 


Materials 

This  group  will  be  given  a  new  set  of  paper  visuals  of  the  engineering  spaces  as  well  as  the 
accompanying  documents. 

Test 

Here  are  some  pictures  of  the  engineering  spaces  on  the  USS  Arleigh  Burke.  You  are  in  port  and 
getting  ready  to  do  a  walk  through  the  spaces  for  a  material  readiness  assessment.  You  have 
some  documents  available  to  you. 

Examine  the  following  areas,  and  answer  the  questions  on  the  answer  sheet  for  each  discrepancy 
you  find.  Some  nodes  may  not  contain  a  discrepancy. 

--  Platform  1,  Nodes  5,  15, 16, 18, 19,  and  21. 

-  Platform  2,  Nodes  5,  10/1 1,  18, 19,  7. 

-  Hold,  Nodes  1,  10, 11,  13, 14,  19. 

You  will  have  one  hour  to  complete  this  part. 
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Control  Group  B  (Visuals  &  Case-based  Instruction) 

Scenario  1 


Materials 


This  group  will  be  given  a  paper  visual  of  the  engineering  spaces  as  well  as  the  accompanying 
documents. 

Method 

An  instructor  will  lead  three  students  through  the  material  assessment  process,  using  pictures. 

Script 

Allow  45  minutes  for  this  part. 

“We  are  looking  into  optimal  ways  of  teaching  material  inspection  to  Engineering  Officers.  You 
have  been  selected  to  try  out  some  new  visual  materials  that  have  been  developed  from  pictures 
taken  aboard  ship.  First  I  will  lead  you  through  the  process  and  then  give  you  some  time  to 
examine  the  pictures  individually.  Then  we  will  give  you  a  new  set  of  similar  pictures  and  ask 
you  some  questions.” 

Step  1.  Preparation  for  material  inspection. 

Instructor  says  to  the  students: 

You  are  in  port  on  an  Arleigh  Burke  class  ship  as  the  CBENG  getting  ready  to  do  a  walk 
through  your  spaces  to  assess  their  material  readiness.  You  have  a  wheelbook  available  to  you. 
At  1200  you  stop  by  control  and  check  the  logs.”  (Logs  available:  Tag  Out,  Out  of  Commission 
(Instrument),  Engineering.) 

“For  example,  from  the  Tagout  Log  you  learn  that  the  MER  2  HFFF  hose  reel  actuation 
button  is  tagged  out  for  repair.”  (Platform  2,  Node  13). 

Instructor  should  model  the  performance  of  the  CHENG  as  s/he  prepares  to  do  a  material 
assessment.  The  instructor  should  explain  (as  it  is  done)  what  s/he  is  looking  for,  what  s/he  finds, 
etc.  Talk  about  making  written  notes  in  the  wheel  book,  prioritizing  discrepancies  at  the  end  of 
the  inspection,  and  making  the  appropriate  entries  in  logs.  The  instructor  should  discuss  tracking 
material  discrepancies:  when  it  is  appropriate  to  put  it  on  the  8  o’clock  report;  when  to  make  a 
note  of  it.  The  instructor  might  also  ask  the  PEOs  questions  about  this  stage  of  the  assessment. 

Step  2. - Instructor  Modeling  of  Material  Inspection  Process.  Modeling  consists  of  a 

demonstration  by  the  instmctor  of  the  process  of  solving  a  problem.  Here  the  instructor  will 
model  how  he  performs  material  inspection. 
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A.  First  inspection  item:  Instructor  says,  “Ordinarily  we  would  proceed  though  the  spaces  in  a 
particular  order.  However,  due  to  time  limits,  for  this  demonstration  we  will  select  certain 
locations  (nodes)  to  examine.” 

Platform  1,  Node  1.  The  instructor  proceeds  to  the  first  zone  where  he  will  start  the  inspection. 
(Explain  that  a  set  of  pictures  were  taken  from  a  certain  location  and  views  were  shot  around  a 
360°  angle.  Explain  the  labels  on  the  pictures:  platform,  node,  image.) 

The  instructor  explains: 

1.  That  he  is  looking  for  something  out  of  the  ordinary  (such  as  corrosion)  or  that  he  is  looking 
for  something  written  up  in  one  of  the  logs.  (Discuss  the  discrepancy  at  Platform  1,  Node  1 — 
AFFE  hose  improperly  stowed.) 

2.  What  actions  he  will  take  with  different  types  of  discrepancies,  i.e.  safety,  housekeeping. 

3.  That  he  will  write  down  what  he  finds,  and  at  the  end  of  the  inspection  he  will  prioritize 
actions  to  be  taken,  using  the  following  categories  (point  these  out  on  the  answer  sheet): 

1 .  Security  breach  or  CAS  REP. 

2.  Fix  immediately. 

3.  Put  on  the  8  o’clock  report — ^fix  this  week. 

4.  Place  discrepancy  on  the  CSMP  or  order  repair  parts. 

5.  Conduct  repairs  or  correct  after  completing  higher  priorities. 

4.  How  this  discrepancy  affects  the  functioning  of  other  equipment  or  systems  aboard  ship. 

5.  What  information  led  him  to  this  situation  assessment  and  these  actions. 

6.  How  s/he  would  follow-up  on  discrepancies  noted  to  see  if  they  had  been  resolved. 

7.  At  the  end  of  the  inspection,  s/he  will  decide  which  discrepancies  to  report  to  the  CO. 

Then  the  instructor  asks  the  students  for  anything  they  might  want  to  add  or  any  questions  they 
might  want  to  ask. 

B.  Second  inspection  item: 

Instructor  goes  to  Platform  2,  Node  14  and  checks  the  breakers.  Instructor  models  the 
inspection  here  using  the  same  procedure  as  used  for  the  first  item.  The  instmctor  asks  the 
following  questions,  “WTiat’s  wrong  with  this?  Why  might  it  be  in  the  off  position?  If  you 
authorized  repairs  on  that  piece  of  equipment  this  morning,  do  you  have  any  concerns  by  seeing 
this?”  (Answer:  Possibly,  if  people  are  not  following  tagout  procedures.  Or  if  they  are  not  doing 
repairs.)  “What  caused  this?”  (Answer:  Safety — equipment  grounded.)  “Is  this  something  you 
want  to  get  fixed  immediately?” 
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— TTiird  inspection  item:  Go  to  Node  15  (Plat.  2).  Look  around.  Say,  “  I  don’t  find  any 
discrepancies  here.” 

Instructor  Coaching.  Coaching  involves  the  instructor  observing  students  as  they  try  to 
complete  tasks  and  providing  hints  and  helps  when  needed. 

A,  Fourth  inspection  item:  Go  to  the  Hold,  Node  16.  The  instructor  tells  the  students  to  look  for 
any  discrepancies.  The  instructor  asks  the  students  what  they  have  found.  (Frayed  cable  to  1 10 
outlet.)  Then  he  asks  them  to  answer  questions  2  through  6  above  regarding  this  item.  If  the 
students  have  trouble,  the  instructor  should  lead  them  to  the  answers  by  asking  them  leading 
questions  to  help  them  discover  the  answer. 

B.  Fifth  inspection  item:  The  instructor  goes  to  Node  4  (Hold)  and  asks  the  student  to  examine 
this  area  for  any  discrepancies.  When  they  find  it,  the  instructor  asks  the  students  to  answer 
questions  2  through  6  above  regarding  this  item.  If  the  students  have  trouble,  the  instructor 
should  lead  them  to  the  answers  by  asking  them  leading  questions  to  help  them  discover  the 
answer.  For  example,  ask  the  student  what  reference  to  access.  Then  the  instructor  asks  the 
student,  “Do  you  see  any  other  discrepancies  in  this  area?”  If  they  find  it,  the  instructor  asks  the 
students  to  answer  questions  2  through  6  above  regarding  this  item.  If  the  students  have  trouble, 
the  instructor  should  lead  them  to  the  answers  by  asking  them  leading  questions  to  help  them 

discover  the  answer.  If  the  students  do  not  find  it,  the  instructor  should  give  them  hints  to  lead 
them  to  it. 

Stev  4.  Student  practice. 

A.  Practice:  The  instructor  says  to  the  students: 

“I  selected  only  a  few  items  for  inspection,  but  now  you  should  examine  the  materials  supplied 
individually  on  your  own  and  look  for  other  discrepancies.  Look  at: 

--  Platform  1,  Nodes  4, 7,  14,  and  23. 

--  Platform  2,  Nodes  4,  8, 13,  6. 

-  Hold,  Nodes  2,  5,  15,  12. 

Some  nodes  may  not  contain  a  discrepancy.  Write  down  your  answers  to  each  of  questions  2 
through  6  above.” 

^^^^en  you  have  completed  the  inspection,  write  down  the  discrepancies  you  would  report  to  the 

Then  the  students  should  be  given  time  to  inspect  several  more  items,  writing  down  their 
selections  and  the  answers  to  questions  2  through  6  above  for  each  item. 

B.  Feedback:  When  the  student  completes  his/her  inspection,  s/he  returns  to  the  instructor  to 
present  the  results  and  get  feedback.  If  items  were  missed,  the  instructor  should  review  the 
documents  with  the  student  and  question  him/her  about  possible  discrepancies  to  investigate. 
Then  the  instructor  should  send  the  student  back  to  look  for  other  discrepancies.  Again,  the 
student  returns  to  the  instructor  and  receives  feedback.  If  additional  items  were  missed,  the 
instructor  should  point  out  the  discrepancies  on  the  visuals  and  ask  the  student  questions  2 
through  6  about  the  items. 
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Control  Group  B  (Visuals  and  Case-based  Instruction) 

Scenario  2 


Materials 

This  group  will  be  given  a  new  set  of  paper  visuals  of  the  engineering  spaces  as  well  as  the 
accompanying  documents. 

Test 

“Here  are  some  pictures  of  the  engineering  spaces  on  the  USS  Arleigh  Burke.  You  are  in  port 
and  getting  ready  to  do  a  walk  through  the  spaces  for  a  material  readiness  assessment.  You  have 
some  documents  available  to  you.” 

“Examine  the  following  areas,  and  answer  the  questions  on  the  answer  sheet  for  each 
discrepancy  you  find.  Some  nodes  may  not  contain  a  discrepancy.” 

--  Platform  1,  Nodes  5,  15, 16,  18, 19,  and  21. 

--  Platform  2,  Nodes  5, 10/11, 18, 19, 7. 

-  Hold,  Nodes  1,  10, 11,  13, 14,  19. 

You  will  have  one  hour  to  complete  this  part. 
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Control  Group  C 

(Virtual  Environment  and  Case-based  Instruction) 

Scenario  1 


1/15/98 


Materials 

This  group  will  see  and  be  able  to  navigate  in  virtual  engineering  spaces  on  the  computer  and 
will  be  given  accompanying  documents. 

Method 

An  instructor  will  lead  three  students  through  the  material  assessment  process,  using  the  virtual 
engineering  spaces  as  depicted  in  the  computer. 

Script 

Allow  45  minutes  for  this  part. 

“We  are  looking  into  optimal  ways  of  teaching  material  inspection  to  Engineering  Officers.  You 
have  been  selected  to  try  out  a  new  depiction  of  the  engineering  spaces  on  a  computer.  First  I 
will  lead  you  through  the  process,  and  then  give  you  some  time  to  try  out  the  computer  system 
individually.  After  that  you  will  be  asked  to  examine  certain  engineering  spaces  on  the  computer 
to  find  discrepancies.” 

Step  1.  Preparation  for  material  inspection. 

Instructor  says  to  the  students: 

You  are  in  port  on  an  Arleigh  Burke  class  ship  as  the  CHENG  getting  ready  to  do  a  walk 
through  your  spaces  to  assess  their  material  readiness.  You  have  a  wheelbook  available  to  you. 
You  stop  by  control  and  check  the  logs.” 

For  example,  from  the  Tagout  Log  you  learn  that  the  MER  2  HFFF  hose  reel  actuation 
button  is  tagged  out  for  repair.”  (Platform  2,  Node  13). 

Instructor  should  model  the  performance  of  the  CHENG  as  s/he  prepares  to  do  a  material 
assessment.  The  instructor  should  explain  (as  it  is  done)  what  s/he  is  looking  for,  what  s/he  finds, 
etc.  Talk  about  making  written  notes  in  the  wheel  book,  prioritizing  discrepancies  at  the  end  of 
the  inspection,  and  making  the  appropriate  entries  in  logs.  The  instructor  should  discuss  tracking 
material  discrepancies;  when  it  is  appropriate  to  put  it  on  the  8  o’clock  report;  when  to  make  a 
note  of  it.  The  instructor  might  also  ask  the  PEOs  questions  about  this  stage  of  the  assessment. 

Step  2.  Instructor  Modelins  of  Material  Inspection  Process.  Modeling  consists  of  a 
demonstration  by  the  instructor  of  the  process  of  solving  a  problem.  Here  the  instructor  will 
model  how  he  performs  material  inspection. 
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A.  First  inspection  item:  Instructor  says,  “Ordinarily  we  would  proceed  though  the  spaces  in  a 
particular  order.  However,  due  to  time  limits,  for  this  demonstration  we  will  select  certain 
locations  (nodes)  to  examine.” 

Platform  1,  Node  1.  The  instructor  proceeds  to  the  first  zone  where  he  will  start  the  inspection. 
The  instructor  explains: 

1 .  That  he  is  looking  for  something  out  of  the  ordinary  (such  as  corrosion)  or  that  he  is  looking 
for  something  written  up  in  one  of  the  logs.  (Discuss  the  discrepancy  at  Platform  1,  Node  1 — 
AFFF  hose  improperly  stowed.) 

2.  What  actions  he  will  take  with  different  types  of  discrepancies,  i.e.  safety,  housekeeping. 

3.  That  he  will  write  down  what  he  finds,  and  at  the  end  of  the  inspection  he  will  prioritize 
actions  to  be  taken,  using  the  following  categories: 

1 .  Security  breach  or  CAS  REP. 

2.  Fix  immediately. 

3.  Put  on  the  8  o’clock  report — ^fix  this  week. 

4.  Place  discrepancy  on  the  CSMP  or  order  repair  parts. 

5.  Conduct  repairs  or  correct  after  completing  higher  priorities. 

4.  How  this  discrepancy  affects  the  functioning  of  other  equipment  or  systems  aboard  ship. 

5-  What  information  led  him  to  this  situation  assessment  and  these  actions. 

6.  Potential  errors  that  less  experienced  engineers  might  make. 

7.  At  the  end  of  the  inspection,  s/he  will  decide  which  discrepancies  to  report  to  the  CO. 

8.  How  s/he  would  follow-up  on  discrepancies  noted  to  see  if  they  had  been  resolved. 

Then  the  instructor  asks  the  students  for  anything  they  might  want  to  add  or  any  questions  they 
might  want  to  ask. 

B.  Second  inspection  item:  Instructor  goes  to  Platform  2,  Node  14  and  checks  the  breakers 
mentioned  in  the  Tagout  log.  Instructor  models  the  inspection  here  using  the  same  procedure  as 
used  for  the  first  item.  The  instmctor  asks  the  following  questions,  “What’s  wrong  with  this? 
Why  might  it  be  in  the  off  position?  If  you  authorized  repairs  on  that  piece  of  equipment  this 
morning,  do  you  have  any  concerns  by  seeing  this?”  (Answer:  Possibly,  if  people  are  not 
following  tagout  procedures.  Or  if  they  are  not  doing  repairs.)  “What  caused  this?”  (Answer: 
Safety — equipment  grounded.)  “Is  this  something  you  want  to  get  fixed  immediately?” 

C.  Third  inspection  item:  Go  to  Node  15  (Plat.  2).  Look  around.  Say,  “  I  don’t  find  any 
discrepancies  here.” 
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Step  3.  Instructor  Coaching.  Coaching  involves  the  instructor  observing  students  as  they  try  to 
complete  tasks  and  providing  hints  and  helps  when  needed. 

Fourth  inspection  item:  Go  to  the  Hold,  Node  16.  The  instructor  tells  the  students  to  find  that 
item.  The  instructor  asks  the  students  what  they  have  found.  (Frayed  cable  to  1 10  outlet.)  Then 
he  asks  them  to  answer  questions  2  through  6  above  regarding  this  item.  If  the  students  have 
trouble,  the  instructor  should  lead  them  to  the  answers  by  asking  them  leading  questions  to  help 
them  discover  the  answer. 

B,  Fifth  inspection  item:  The  instructor  goes  to  Node  4  (Hold)  and  asks  the  student  to  examine 
this  area  for  any  discrepancies.  When  they  find  it,  the  instructor  asks  the  students  to  answer 
questions  2  through  6  above  regarding  this  item.  If  the  students  have  trouble,  the  instructor 
should  lead  them  to  the  answers  by  asking  them  leading  questions  to  help  them  discover  the 
answer.  For  example,  ask  the  student  what  reference  to  access.  Then  the  instructor  asks  the 
student,  “Do  you  see  any  other  discrepancies  in  this  area?”  If  they  find  it,  the  instructor  asks  the 
students  to  answer  questions  2  through  6  above  regarding  this  item.  If  the  students  have  trouble, 
the  instructor  should  lead  them  to  the  answers  by  asking  them  leading  questions  to  help  them 

discover  the  answer.  If  the  students  do  not  find  it,  the  instructor  should  give  them  hints  to  lead 
them  to  it. 

Step  4.  Student  practice. 

A.  Practice:  The  instructor  says  to  the  students: 

“I  selected  only  a  few  items  for  inspection,  but  now  you  should  examine  the  engineering  spaces 
on  the  computer  individually  on  your  own  and  look  for  other  discrepancies.  Look  at: 

--  Platform  1,  Nodes  4,  7,  14,  and  23. 

--  Platform  2,  Nodes  4,  8,  13,  6. 

—  Hold,  Nodes  2, 5,  15,  12. 

Some  nodes  may  not  contain  discrepancy.  Write  down  your  answers  to  each  of  questions  2 
through  6  above.  When  you  have  completed  the  inspection,  write  down  the  discrepancies  you 
would  report  to  the  CO.  Then  the  students  should  be  given  time  to  inspect  several  more  items, 
writing  down  their  selections  and  the  answers  to  questions  2  through  6  above  for  each  item. 

— •  Bssdback:  When  the  student  completes  his/her  inspection,  s/he  returns  to  the  instructor  to 
present  the  results  and  get  feedback.  If  items  were  missed,  the  instructor  should  review  the 
documents  with  the  student  and  question  him/her  about  possible  discrepancies  to  investigate. 
Then  the  instructor  should  send  the  student  back  to  look  for  other  discrepancies.  Again,  the 
student  returns  to  the  instructor  and  receives  feedback.  If  additional  items  were  missed,  the 
instructor  should  point  out  the  discrepancies  on  the  computer  and  ask  the  student  questions  2 
through  6  about  the  items. 
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Control  Group  C 

(Virtual  Environment  &  Case-based  Instruction) 

Scenario  2 


Materials 

This  group  will  be  given  a  new  list  of  locations  on  the  computer  to  examine;  accompanying 
documents  will  be  provided. 

Test 

“Using  the  computer  scenes  depicting  the  engineering  spaces  on  the  USS  Arleigh  Burke  (in 
port),,  you  are  getting  ready  to  do  a  walk  through  the  spaces  for  a  material  readiness  assessment. 
You  have  some  documents  available  to  you.” 

“Examine  the  following  areas,  and  answer  the  questions  on  the  answer  sheet  for  each  discrepancy 
you  find.  Some  nodes  may  not  contain  a  discrepancy.” 

--  Platform  1,  Nodes  5,  15,  16,  18,  19,  and  21. 

-  Platform  2,  Nodes  5,  10/1 1,  18, 19, 7. 

-  Hold,  Nodes  1, 10, 11,  13, 14,  19. 

You  will  have  one  hour  to  complete  this  part. 
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Appendix  D 

Equipment  Requirements 
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Hardward  and  Software  Requirements 

Digital  Image-Based  System 

The  image-based  application  rans  on  a  conventional  pentium  class  desktop  computer.  The 
minimum  requirements  are: 


Hardware 

Software 

166Mhz  MMX  Pentium  CPU 

Windows  95 

Accelerated  SVGA  graphics  card  with  2MB  of  video  ram 

QuickTime  for  Windows 

17”  .28  SVGA  non-interlaced  monitor  (21”  optional) 

2.1  GB  Hard  Drive 

32MB  Ram 

256KB  L2  cache 

16X  CD-ROM 

Model-Based  System 

The  model-based  application  mns  on  a  pentium  n  desktop  computer  in  a  Windows  NT 
environment.  The  minimum  requirements  are: 


Hardware 

Software 

300  Mhz  Pentium  11  CPU 

Windows  NT  4.0 

Intel  AL440LX  chipset  motherboard  or  equivalent 

World  Tool  Kit 

128  MB  Ram 

Open  GL  Drivers 

512K  Pipe  cache 

Open  GL  3D  graphics  and  texture  accelerator  for 

visualization  and  animation  (Diamond  Fire  GL  3000 
board  with  8MB  of  VRAM  and  32MB  of  DRAM  or 
equivalent) 

17”  .28  SVGA  non-interlaced  monitor  (21”  optional) 

4.3  GB  Hard  Drive 

24X  CD-ROM 

Sound  Blaster  compatible  sound  card  with  .wav  capability 
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Draft  Student  VET-E  Survey 


Appendix  E 

Surface  Warfare  Officer  School  Student  Questionnaire 
Of  Virtual  Environment  Training 
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Draft  Student  VET-E  Survey 


Surface  Warfare  Officer  School  Student  Survey 
of  Virtual  Environment  Training 


Please  indicate  extent  to  which  you  agree  or  disagree  with  the  following  statements.  Circle  the 
appropriate  letter  on  the  form  below. 


Greatly 

Somewhat 

Somewhat 

Greatly 

Doesn’t 

Disagree 

Disagree  Uncertain 

Agree 

Agree 

Apply 

L 

It  was  easy  to  “move”  through  the 
propulsion  plant  with  the  approach  I 
was  given. 

A 

B 

c 

D 

E 

F 

2. 

I  was  able  to  view  the  scenes  fairly 
quickly. 

A 

B 

c 

D 

E 

F 

3. 

Many  of  the  images  were  unclear. 

A 

B 

c 

D 

E 

F 

4. 

I  had  to  spend  too  much  time  going 
back-and-forth  within  a  node  to  find 
a  discrepancy. 

A 

B 

c 

D 

E 

F 

5. 

The  propulsion  plant  scenes  were 
arranged  in  a  logical  order. 

A 

B 

c 

D 

E 

F 

6. 

The  viewing  area  in  the  propulsion 
plant  was  too  restricted  to  be  useful. 

A 

B 

c 

D 

E 

F 

7. 

I  was  able  to  learn  about  material 
readiness  assessment  using  the 
approach  I  was  given. 

A 

B 

c 

D 

E 

F 

8. 

It  was  difficult  to  figure  out  where  I 
was  in  the  propulsion  plant. 

A 

B 

c 

D 

E 

F 

9. 

This  approach  to  identify  dis¬ 
crepancies  was  confusing. 

A 

B 

c 

D 

E 

F 

10. 

The  discrepancy  identification  pro¬ 
cess  was  confusing. 

A 

B 

c 

D 

E 

F 

11. 

It  was  fairly  easy  to  move  from 
none-to-node. 

A 

B 

c 

D 

E 

F 

12. 

It  was  easy  to  move  from  platform- 
to-platform. 

A 

B 

c 

D 

E 

F 

13. 

There  should  be  an  easier  way  to 
navigate  through  the  engine  room 
spaces. 

A 

B 

c 

D 

E 

F 

14. 

The  fidelity  or  clarity  of  the  pro¬ 
pulsion  plant  was  good. 

A 

B 

c 

D 

E 

F 

15. 

It  was  difficult  to  clearly  see  any 
discrepancies. 

A 

B 

c 

D 

E 

F 

16. 

It  was  difficult  to  distinguish 

A 

B 

c 

D 

E 

F 

between  platforms. 
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Draft  Student  VET-E  Survey 


Greatly 

Disagree 

Somewhat 

Disagree 

Uncertain 

Somewhat 

Agree 

Greatly 

Agree 

Doesn’t 

AddIv 

17.  The  views  of  the  propulsion  plant 
were  very  accurate. 

A 

B 

c 

D 

E 

F 

18.  I  have  a  better  understanding  than  I 
had  before  of  what  the  propulsion 
plant  of  the  ARLEIGH  BURKE 
class  of  ships  looks  like. 

A 

B 

c 

D 

E 

F 

19.  The  instructions  I  was  given  were 
helpful  in  understanding  my  task. 

A 

B 

c 

D 

E 

F 

20.  The  instructions  I  was  given  were 
helpful  in  “navigating”  through  the 
spaces. 

A 

B 

c 

D 

E 

F 

21.  This  approach  to  learning  about 
material  discrepancies  was  useful. 

A 

B 

c 

D 

E 

F 

22.  This  approach  should  be  a  bigger 
part  of  the  course  training  process. 

A 

B 

c 

D 

E 

F 

23.  Compared  to  the  typical  classroom 
instruction,  the  present  approach 
used  to  examine  the  engineering 
spaces  helped  me  to  understand  the 
material  assessment  process  better. 

A 

B 

c 

D 

E 

F 

24.  The  approach  used  by  the  instructor 
is  one  that  is  also  used  in  the 
classroom. 

A 

B 

c 

D 

E 

F 

25.  The  information  given  by  the 
instructor  was  clear. 

A 

B 

c 

D 

E 

F 

26.  In  general,  I  was  satisfied  using  this 
approach  to  learn  about  the  material 
discrepancy  process. 

A 

B 

c 

D 

E 

F 

27.  In  genereil,  I  was  satisfied  using  this 
approach  to  identify  discrepancies. 

A 

B 

c 

D 

E 

F 

28.  In  general,  I  do  not  think  this 
approach  for  identifying  material 
discrepancies  is  a  good  one. 

A 

B 

c 

D 

E 

F 

29.  This  form  of  instruction  was  useful 
compared  to  the  traditional  forms  of 
classroom  instruction. 

A 

B 

c 

D 

E 

F 

30.  Compared  to  the  typical  classroom 
instruction,  the  present  approach 
helped  me  to  understand  how  the 
equipment  worked  together  better. 

A 

B 

c 

D 

E 

F 

31.  Compared  to  the  typical  classroom 
instruction,  the  present  approach 

A 

B 

c 

D 

E 

F 

used  to  examine  the  engineering 
spaces  helped  me  to  visualize  the 
work  spaces  better. 
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Draft  Student  VET-E  Survey 


Please  provide  the  following  information: 

What  is  your  rank? _ 

How  long  have  you  been  in  the  Navy?  _ 

How  many  tours  have  you  had  at  sea? _ 

How  many  years  have  you  served  at  sea? _ 

How  many  ships  have  you  served  aboard? _ 

What  types  and  how  much  engineering  training  have  you  had  (e.g.,  education,  OJT)? 

College _ _ 

Navy _ 

OJT _ 

Civilian _ 

Other  (specify) _ 

What  engineering  related  jobs  have  you  held  aboard  ship  and  for  long? 

On  your  most  recent  tour,  you  served  as  a: _ 

On  your  next  tour,  you  will  serve  as  a:  _ 

How  much  experience  have  you  had  using  computers? 

None 

Six  months  or  less 
Six  months  to  1  year 
1  to  3  years 
More  than  3  years 
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Surface  Warfare  Officer  School  Instructor  Questionnaire 
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Surface  Warfare  Officer  School  Instructor 
Survey  of  Virtual  Environment  Training 

Please  indicate  extent  to  which  you  agree  or  disagree  with  the  following  statements.  Circle 
the  appropriate  letter  on  the  form  below. 


Greatly 

Disagree 

Somewhat 

Disagree 

Uncertain 

Somewhat 

Agree 

Greatly 

Agree 

Doesn’t 

Agree 

1 .  I  like  this  approach  to  teach 
about  the  material  condi¬ 
tion  of  the  propulsion  plant. 

A 

B 

c 

D 

E 

F 

2.  It  was  easy  for  the  students 
to  “move”  through  the  pro¬ 
pulsion  plant  with  the 
approach  I  used. 

A 

B 

c 

D 

E 

F 

3.  Many  of  the  images  were 
unclear. 

A 

B 

c 

D 

E 

F 

4.  I  had  to  spend  too  much 
time  clarifying  information 
for  the  students. 

A 

B 

c 

D 

E 

F 

5.  The  propulsion  plant  scenes 
were  arranged  in  a  logical 
order. 

A 

B 

c 

D 

E 

F 

6.  The  viewing  area  in  the 
propulsion  plant  was  too 
restricted  to  be  useful. 

A 

B 

c 

D 

E 

F 

7.  The  students  were  able  to 
learn  about  material  readi¬ 
ness  assessment  using  the 
approach  I  used. 

A 

B 

c 

D 

E 

F 

8.  The  present  approach 
added  information  that 
otherwise  would  be  dif¬ 
ficult  to  include  to  under¬ 
stand  the  material  condition 
of  the  propulsion  plant. 

A 

B 

c 

D 

E 

F 

9.  This  approach  to  identify 
discrepancies  was  useful. 

A 

B 

c 

D 

E 

F 

10.  The  discrepancy  identifica¬ 
tion  process  used  in  this 
approach  was  confusing. 

A 

B 

c 

D 

E 

F 

11.  It  was  fairly  easy  for 
students  to  move  from 
none-to-node. 

A 

B 

c 

D 

E 

F 
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12.  It  was  fairly  easy  to  move 
from  platfom-to-platform. 

13.  This  approach  provided  a 
useful  way  to  teach  about 
material  discrepancies. 

14.  The  fidelity  or  clarity  of  the 
images  of  the  propulsion 
plant  were  good. 

15.  It  was  difficult  for  students 
to  clearly  see  discrepancies. 

16.  It  was  difficult  to  distin¬ 
guish  between  platforms. 

17.  The  views  of  the  propulsion 
plant  were  accurate. 

18.  The  present  approach 
should  give  a  good  per¬ 
spective  of  what  the 
propulsion  plant  of  the 
ARLEIGH  BURKE  class 
of  ships  looks  like. 

19.  The  present  approach  did 
not  add  useful  information 
beyond  what  is  presently 
taught  in  the  classroom. 

20.  The  present  approach  will 
detract  from  students  learn¬ 
ing  about  the  propulsion 
plant  operations. 

21.  The  present  approach  has 
applications  beyond  its  use 
in  the  propulsion  plant. 

22.  This  approach  should  be  a 
bigger  part  of  the  course 
training  process. 

23.  Compared  to  the  typical 
classroom  instruction,  the 
present  approach  used  to 
examine  the  engineering 
spaces  will  help  students 
understand  the  material 
assessment  process  better. 

24.  Compared  to  the  typical 
classroom  instruction,  the 
present  approach  used  to 
examine  the  engineering 
spaces  will  be  more 
confusing  to  the  students. 


Greatly  Somewhat 
Disagree  Disagree  Uncertain 


ABC 

ABC 

ABC 

ABC 

ABC 

ABC 

ABC 


ABC 

ABC 

ABC 

ABC 

ABC 


ABC 


Somewhat 

Agree 

Greatly 

Agree 

Doesn’t 

Agree 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

D 

E 

F 

Greatly 

Disagree 

Somewhat 

Disagree 

Uncertain 

Somewhat 

Agree 

Greatly 

Agree 

DoesnH 

Agree 

25.  In  general,  I  was  satisfied 
using  this  approach  to  teach 
about  the  material  discrep¬ 
ancy  process. 

A 

B 

c 

D 

E 

F 

26.  This  approach  required  me 
to  spend  more  time  to 
prepare  for  my  students. 

A 

B 

c 

D 

E 

F 

27.  In  general,  I  do  not  think 
this  approach  for  identi¬ 
fying  material  discrep¬ 
ancies  is  a  good  one. 

A 

B 

c 

D 

E 

F 

28.  Compared  to  the  traditional 
classroom  instruction,  I 
would  recommend  this 
approach  as  an  important 
supplement. 

A 

B 

c 

D 

E 

F 

29.  Compared  to  the  traditional 
classroom  instruction,  the 
present  approach  used  to 
examine  the  engineering 
spaces  required  me  to 
change  my  teaching  style. 

A 

B 

c 

D 

E 

F 

30.  The  present  approach  used 

A 

B 

c 

D 

E 

F 

to  examine  the  engineering 
spaces  will  help  students 
when  they  serve  as  a 
CHENG. 
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Please  provide  the  following  information; 

What  is  your  rank? _ _ 

How  long  have  you  been  in  the  Navy? 

How  many  years  have  you  served  at  sea?  _ 

How  many  ships  have  you  served  aboard? _ 

How  long  have  you  been  an  instructor  at  SWOS?  _ _ at  other  locations? 

What  types  and  how  much  engineering  training  have  you  had  (e.g.,  education,  OJT)? 

College _ _ 

Navy _ _ _ 

OJT _ _ _ _ 

Civilian _ _ _ 

Other  (specify) _ _ _ _ 

What  engineering  related  jobs  have  you  held  aboard  ship  and  for  long? 

On  your  most  recent  tour,  you  served  as  a: _ 

On  your  next  tour,  you  will  serve  as  a:  _ 

How  much  experience  have  you  had  using  computers? 

None 

Six  months  or  less 
Six  months  to  1  year 
1  to  3  years 
More  than  3  years 
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